Result: 264 Warning(s)
i915_display_info5 igt_runner5 results5.json results5-xe-load.json guc_logs5.tar i915_display_info_post_exec5 serial_data5 boot5 dmesg5
| Detail | Value |
|---|---|
| Duration | unknown |
| Hostname |
shard-bmg-8 |
| Igt-Version |
IGT-Version: 2.4-gb024a3b67 (x86_64) (Linux: 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ x86_64) |
| Out |
Using IGT_SRANDOM=1779903857 for randomisation Opened device: /dev/dri/card0 Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298: Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10) Last errno: 2, No such file or directory Starting subtest: pt-caching va_bits: 48, num_objs: 16384, spread over pt levels: [0]: 16384 [1]: 512 [2]: 16 Subtest pt-caching: SUCCESS (7.952s) This test caused an abort condition: Kernel badly tainted (0x244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened. |
| Err |
Starting subtest: pt-caching Subtest pt-caching: SUCCESS (7.952s) |
| Dmesg |
<6> [512.725904] Console: switching to colour dummy device 80x25
<6> [512.726207] [IGT] xe_pat: executing
<6> [512.766267] [IGT] xe_pat: starting subtest pt-caching
<6> [514.626765] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x0000000001790000
FaultType: 0
AccessType: 0
FaultLevel: 0
EngineClass: 3 bcs
EngineInstance: 8
<6> [514.626777] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<6> [514.626825] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<6> [514.627318] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [514.627328] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61008, lrc_seqno=61008, guc_id=0, flags=0x73 in no process [-1]
<7> [514.627332] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.627450] ------------[ cut here ]------------
<4> [514.627452] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.627453] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:21/4151
<4> [514.627545] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.627627] [last unloaded: snd_hda_intel]
<4> [514.627631] CPU: 4 UID: 0 PID: 4151 Comm: kworker/u64:21 Tainted: G S U 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.627635] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER
<4> [514.627636] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.627638] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.627645] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.627733] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.627735] RSP: 0018:ffffc90005e57ca0 EFLAGS: 00010246
<4> [514.627738] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.627739] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.627741] RBP: ffffc90005e57db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.627742] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.627743] R13: ffff888104285410 R14: ffff888299690000 R15: 00000000fffffffb
<4> [514.627744] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [514.627746] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.627747] CR2: 00007f623e4edfd0 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [514.627749] PKRU: 55555554
<4> [514.627750] Call Trace:
<4> [514.627752] <TASK>
<4> [514.627756] ? lock_acquire+0xb0/0x300
<4> [514.627763] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [514.627768] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.627774] process_one_work+0x239/0x740
<4> [514.627782] worker_thread+0x200/0x3f0
<4> [514.627785] ? __pfx_worker_thread+0x10/0x10
<4> [514.627788] kthread+0x10d/0x150
<4> [514.627790] ? __pfx_kthread+0x10/0x10
<4> [514.627793] ret_from_fork+0x3bd/0x470
<4> [514.627796] ? __pfx_kthread+0x10/0x10
<4> [514.627799] ret_from_fork_asm+0x1a/0x30
<4> [514.627807] </TASK>
<4> [514.627808] irq event stamp: 430841
<4> [514.627810] hardirqs last enabled at (430847): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.627816] hardirqs last disabled at (430854): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.627818] softirqs last enabled at (427466): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.627822] softirqs last disabled at (427461): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.627825] ---[ end trace 0000000000000000 ]---
<6> [514.627827] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.627912] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.627980] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.628255] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.628971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.629092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.629199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.629300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.629397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.629490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.629583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.629677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.629766] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.629884] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.629988] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.631198] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.641819] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.642088] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.643753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.643865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.643959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.644051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.644141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.644227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.644311] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.644394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.644476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.644559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.644641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.644722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.644802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.644896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.644981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.645063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.645145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.645226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.645305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.645398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.645490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.645577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.645662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.645749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.645837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.645921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.646004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.646086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.646169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.646255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.646341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.646425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.646510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.646595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.646685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.646775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.646871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.646958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.647047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.647132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.647217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.647299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.647380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.647464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.647549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.647632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.647715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.647797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.647888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.647972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.648056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.648138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.648220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.648301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.648382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.648464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.648546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.648626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.648707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.648801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.648894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.648977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.649059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.649138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.649220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.649301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.649382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.649462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.649549] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.650507] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.650512] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61008, lrc_seqno=61008, guc_id=0, flags=0x73 in no process [-1]
<7> [514.650515] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.650598] ------------[ cut here ]------------
<4> [514.650599] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.650600] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:21/4151
<4> [514.650692] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.650760] [last unloaded: snd_hda_intel]
<4> [514.650764] CPU: 4 UID: 0 PID: 4151 Comm: kworker/u64:21 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.650767] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.650768] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.650769] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.650776] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.650872] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.650874] RSP: 0018:ffffc90005e57ca0 EFLAGS: 00010246
<4> [514.650877] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.650879] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.650880] RBP: ffffc90005e57db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.650881] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.650883] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.650884] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [514.650886] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.650887] CR2: 00007f623e4edfd0 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [514.650888] PKRU: 55555554
<4> [514.650890] Call Trace:
<4> [514.650891] <TASK>
<4> [514.650895] ? lock_acquire+0xb0/0x300
<4> [514.650901] ? lock_release+0xd0/0x2b0
<4> [514.650906] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.650912] process_one_work+0x239/0x740
<4> [514.650920] worker_thread+0x200/0x3f0
<4> [514.650923] ? __pfx_worker_thread+0x10/0x10
<4> [514.650926] kthread+0x10d/0x150
<4> [514.650929] ? __pfx_kthread+0x10/0x10
<4> [514.650932] ret_from_fork+0x3bd/0x470
<4> [514.650935] ? __pfx_kthread+0x10/0x10
<4> [514.650937] ret_from_fork_asm+0x1a/0x30
<4> [514.650946] </TASK>
<4> [514.650947] irq event stamp: 434195
<4> [514.650948] hardirqs last enabled at (434201): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.650951] hardirqs last disabled at (434206): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.650953] softirqs last enabled at (434034): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.650957] softirqs last disabled at (434029): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.650959] ---[ end trace 0000000000000000 ]---
<6> [514.650961] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.651048] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.651056] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.652267] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.652392] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.652608] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.652845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.652958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.653062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.653158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.653251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.653340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.653426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.653514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.653597] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.653699] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.653793] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.655103] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.665821] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.666084] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.667248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.667337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.667419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.667499] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.667580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.667657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.667735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.667819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.667915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.668000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.668078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.668154] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.668230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.668306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.668381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.668456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.668530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.668604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.668678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.668764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.668865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.668961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.669042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.669126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.669209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.669289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.669367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.669444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.669522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.669604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.669685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.669765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.669862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.669959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.670047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.670134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.670220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.670300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.670380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.670458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.670536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.670612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.670689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.670768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.670862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.670953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.671032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.671109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.671186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.671265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.671361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.671441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.671520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.671597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.671671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.671745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.671833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.671927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.672009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.672091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.672171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.672250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.672328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.672405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.672487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.672568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.672646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.672724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.672805] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.673118] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.673124] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61008, lrc_seqno=61008, guc_id=0, flags=0x73 in no process [-1]
<7> [514.673127] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.673196] ------------[ cut here ]------------
<4> [514.673198] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.673199] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:13/2860
<4> [514.673280] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.673349] [last unloaded: snd_hda_intel]
<4> [514.673353] CPU: 10 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.673356] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.673358] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.673359] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.673365] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.673442] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.673444] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.673446] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.673448] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.673449] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.673450] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.673451] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.673453] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [514.673455] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.673456] CR2: 00007df35c067268 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.673457] PKRU: 55555554
<4> [514.673459] Call Trace:
<4> [514.673460] <TASK>
<4> [514.673464] ? lock_acquire+0xb0/0x300
<4> [514.673470] ? lock_release+0xd0/0x2b0
<4> [514.673475] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.673481] process_one_work+0x239/0x740
<4> [514.673489] worker_thread+0x200/0x3f0
<4> [514.673492] ? __pfx_worker_thread+0x10/0x10
<4> [514.673495] kthread+0x10d/0x150
<4> [514.673497] ? __pfx_kthread+0x10/0x10
<4> [514.673500] ret_from_fork+0x3bd/0x470
<4> [514.673502] ? __pfx_kthread+0x10/0x10
<4> [514.673505] ret_from_fork_asm+0x1a/0x30
<4> [514.673513] </TASK>
<4> [514.673514] irq event stamp: 1767283
<4> [514.673515] hardirqs last enabled at (1767289): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.673518] hardirqs last disabled at (1767294): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.673520] softirqs last enabled at (1766286): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.673523] softirqs last disabled at (1766279): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.673525] ---[ end trace 0000000000000000 ]---
<5> [514.674260] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61063, lrc_seqno=61063, guc_id=0, flags=0x73 in no process [-1]
<7> [514.674269] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.674594] ------------[ cut here ]------------
<4> [514.674597] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.674600] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:13/2860
<4> [514.674761] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package
<7> [514.674825] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [514.675039] snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [514.675090] CPU: 11 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.675094] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.675097] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.675099] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [514.675044] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [514.675108] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.675224] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.675226] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.675229] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.675230] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.675231] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.675233] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.675234] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.675235] FS: 0000000000000000(0000) GS:ffff8888db003000(0000) knlGS:0000000000000000
<4> [514.675237] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.675238] CR2: 0000727e754d7098 CR3: 000000010f264002 CR4: 0000000000f72ef0
<4> [514.675240] PKRU: 55555554
<4> [514.675241] Call Trace:
<4> [514.675242] <TASK>
<4> [514.675246] ? lock_acquire+0xb0/0x300
<4> [514.675251] ? lock_release+0xd0/0x2b0
<4> [514.675256] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.675262] process_one_work+0x239/0x740
<4> [514.675269] worker_thread+0x200/0x3f0
<4> [514.675272] ? __pfx_worker_thread+0x10/0x10
<4> [514.675275] kthread+0x10d/0x150
<4> [514.675277] ? __pfx_kthread+0x10/0x10
<4> [514.675280] ret_from_fork+0x3bd/0x470
<4> [514.675283] ? __pfx_kthread+0x10/0x10
<4> [514.675286] ret_from_fork_asm+0x1a/0x30
<4> [514.675294] </TASK>
<4> [514.675295] irq event stamp: 1768817
<4> [514.675296] hardirqs last enabled at (1768823): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.675298] hardirqs last disabled at (1768828): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.675300] softirqs last enabled at (1766286): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.675303] softirqs last disabled at (1766279): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.675306] ---[ end trace 0000000000000000 ]---
<6> [514.675308] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.675389] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.675395] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.675645] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.675866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.675966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.676064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.676155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.676243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.676331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.676417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.676505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.676588] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.676689] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.676785] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.677841] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.687821] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.688081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.689150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.689236] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.689317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.689397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.689474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.689550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.689626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.689702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.689778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.689866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.689946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.690021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.690096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.690172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.690248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.690323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.690399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.690474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.690549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.690638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.690722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.690803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.690899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.690979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.691058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.691135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.691212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.691289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.691365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.691445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.691526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.691604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.691684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.691763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.691861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.691946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.692032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.692112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.692191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.692270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.692347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.692424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.692500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.692575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.692653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.692729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.692806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.692907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.692986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.693067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.693150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.693227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.693305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.693382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.693458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.693535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.693613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.693689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.693765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.693853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.693929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.694006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.694082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.694157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.694234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.694310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.694385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.694459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.694537] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.694659] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.694663] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61063, lrc_seqno=61063, guc_id=0, flags=0x73 in no process [-1]
<7> [514.694666] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.694734] ------------[ cut here ]------------
<4> [514.694735] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.694736] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:13/2860
<4> [514.694821] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.694887] [last unloaded: snd_hda_intel]
<4> [514.694891] CPU: 11 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.694894] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.694895] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.694896] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.694902] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.694978] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.694979] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.694982] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.694983] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.694985] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.694986] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.694987] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.694988] FS: 0000000000000000(0000) GS:ffff8888db003000(0000) knlGS:0000000000000000
<4> [514.694990] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.694991] CR2: 0000727e754d7098 CR3: 000000010f264002 CR4: 0000000000f72ef0
<4> [514.694993] PKRU: 55555554
<4> [514.694994] Call Trace:
<4> [514.694995] <TASK>
<4> [514.694999] ? lock_acquire+0xb0/0x300
<4> [514.695004] ? lock_release+0xd0/0x2b0
<4> [514.695009] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.695015] process_one_work+0x239/0x740
<4> [514.695022] worker_thread+0x200/0x3f0
<4> [514.695025] ? __pfx_worker_thread+0x10/0x10
<4> [514.695027] kthread+0x10d/0x150
<4> [514.695029] ? __pfx_kthread+0x10/0x10
<4> [514.695032] ret_from_fork+0x3bd/0x470
<4> [514.695035] ? __pfx_kthread+0x10/0x10
<4> [514.695037] ret_from_fork_asm+0x1a/0x30
<4> [514.695045] </TASK>
<4> [514.695046] irq event stamp: 1772001
<4> [514.695047] hardirqs last enabled at (1772007): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.695050] hardirqs last disabled at (1772012): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.695051] softirqs last enabled at (1771026): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.695054] softirqs last disabled at (1771019): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.695057] ---[ end trace 0000000000000000 ]---
<6> [514.695849] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.696042] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.696053] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.696092] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.696206] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.696322] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.696589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.696738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.696902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.697002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.697095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.697189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.697278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.697368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.697451] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.697551] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.697645] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.698684] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.708820] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.709076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.710826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.710934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.711028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.711117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.711203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.711287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.711370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.711452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.711534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.711617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.711698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.711777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.711874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.711956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.712036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.712117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.712197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.712278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.712359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.712452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.712544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.712630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.712716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.712802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.712900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.712985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.713069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.713152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.713235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.713320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.713405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.713489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.713574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.713660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.713750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.713847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.713940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.714028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.714113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.714196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.714280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.714363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.714444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.714527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.714615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.714698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.714782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.714872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.714954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.715036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.715121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.715204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.715292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.715374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.715457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.715541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.715625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.715706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.715787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.715879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.715962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.716045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.716127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.716206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.716288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.716370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.716450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.716529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.716623] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.716773] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.716778] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61063, lrc_seqno=61063, guc_id=0, flags=0x73 in no process [-1]
<7> [514.716780] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.716863] ------------[ cut here ]------------
<4> [514.716864] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.716866] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [514.716950] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.717017] [last unloaded: snd_hda_intel]
<4> [514.717021] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.717024] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.717025] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.717027] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.717033] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.717114] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.717116] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.717118] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.717119] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.717121] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.717122] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.717123] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.717125] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.717126] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.717128] CR2: 0000727e7460d098 CR3: 000000012f1a0003 CR4: 0000000000f72ef0
<4> [514.717129] PKRU: 55555554
<4> [514.717131] Call Trace:
<4> [514.717132] <TASK>
<4> [514.717136] ? lock_acquire+0xb0/0x300
<4> [514.717141] ? lock_release+0xd0/0x2b0
<4> [514.717146] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.717153] process_one_work+0x239/0x740
<4> [514.717160] worker_thread+0x200/0x3f0
<4> [514.717163] ? __pfx_worker_thread+0x10/0x10
<4> [514.717166] kthread+0x10d/0x150
<4> [514.717168] ? __pfx_kthread+0x10/0x10
<4> [514.717171] ret_from_fork+0x3bd/0x470
<4> [514.717174] ? __pfx_kthread+0x10/0x10
<4> [514.717177] ret_from_fork_asm+0x1a/0x30
<4> [514.717185] </TASK>
<4> [514.717186] irq event stamp: 332251
<4> [514.717187] hardirqs last enabled at (332257): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.717189] hardirqs last disabled at (332262): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.717191] softirqs last enabled at (331096): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.717194] softirqs last disabled at (331089): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.717197] ---[ end trace 0000000000000000 ]---
<5> [514.717841] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61118, lrc_seqno=61118, guc_id=0, flags=0x73 in no process [-1]
<7> [514.717850] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.718207] ------------[ cut here ]------------
<4> [514.718209] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.718212] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#9: kworker/u64:45/4176
<4> [514.718543] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.718620] [last unloaded: snd_hda_intel]
<4> [514.718624] CPU: 9 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.718627] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.718628] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.718630] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.718636] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<7> [514.718662] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [514.718757] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.718760] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.718762] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.718764] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.718766] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.718767] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.718769] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.718770] FS: 0000000000000000(0000) GS:ffff8888daf03000(0000) knlGS:0000000000000000
<4> [514.718772] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.718774] CR2: 0000727e7401b098 CR3: 000000000364a002 CR4: 0000000000f72ef0
<4> [514.718776] PKRU: 55555554
<4> [514.718777] Call Trace:
<4> [514.718778] <TASK>
<4> [514.718786] ? lock_acquire+0xb0/0x300
<4> [514.718794] ? lock_release+0xd0/0x2b0
<4> [514.718802] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.718831] process_one_work+0x239/0x740
<4> [514.718843] worker_thread+0x200/0x3f0
<4> [514.718848] ? __pfx_worker_thread+0x10/0x10
<4> [514.718853] kthread+0x10d/0x150
<4> [514.718856] ? __pfx_kthread+0x10/0x10
<4> [514.718860] ret_from_fork+0x3bd/0x470
<4> [514.718863] ? __pfx_kthread+0x10/0x10
<4> [514.718867] ret_from_fork_asm+0x1a/0x30
<4> [514.718880] </TASK>
<4> [514.718882] irq event stamp: 334143
<4> [514.718884] hardirqs last enabled at (334149): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.718887] hardirqs last disabled at (334154): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.718890] softirqs last enabled at (334088): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.718894] softirqs last disabled at (334081): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.718898] ---[ end trace 0000000000000000 ]---
<7> [514.718835] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [514.718901] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.719035] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.719043] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.719248] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.719470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.719571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.719670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.719760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.719856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.719943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.720030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.720117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.720201] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.720303] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.720399] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.721454] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.731818] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.732078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.733152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.733264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.733359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.733450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.733537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.733622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.733708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.733794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.733889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.733973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.734055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.734136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.734217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.734297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.734378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.734458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.734538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.734618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.734698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.734791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.734895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.734982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.735067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.735154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.735240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.735322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.735405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.735488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.735570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.735656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.735741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.735828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.735912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.735996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.736086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.736175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.736265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.736351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.736437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.736521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.736605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.736689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.736771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.736862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.736946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.737029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.737112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.737193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.737275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.737356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.737440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.737521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.737604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.737685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.737766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.737856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.737938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.738019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.738100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.738184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.738266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.738349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.738430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.738511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.738592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.738673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.738754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.738839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.738923] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.739050] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.739054] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61118, lrc_seqno=61118, guc_id=0, flags=0x73 in no process [-1]
<7> [514.739057] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.739130] ------------[ cut here ]------------
<4> [514.739131] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.739132] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [514.739216] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.739283] [last unloaded: snd_hda_intel]
<4> [514.739286] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.739289] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.739290] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.739292] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.739298] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.739379] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.739381] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.739383] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.739385] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.739386] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.739387] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.739388] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.739390] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.739391] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.739393] CR2: 0000727e7460d098 CR3: 000000012f1a0003 CR4: 0000000000f72ef0
<4> [514.739394] PKRU: 55555554
<4> [514.739395] Call Trace:
<4> [514.739397] <TASK>
<4> [514.739401] ? lock_acquire+0xb0/0x300
<4> [514.739406] ? lock_release+0xd0/0x2b0
<4> [514.739411] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.739418] process_one_work+0x239/0x740
<4> [514.739425] worker_thread+0x200/0x3f0
<4> [514.739428] ? __pfx_worker_thread+0x10/0x10
<4> [514.739431] kthread+0x10d/0x150
<4> [514.739433] ? __pfx_kthread+0x10/0x10
<4> [514.739436] ret_from_fork+0x3bd/0x470
<4> [514.739438] ? __pfx_kthread+0x10/0x10
<4> [514.739441] ret_from_fork_asm+0x1a/0x30
<4> [514.739449] </TASK>
<4> [514.739450] irq event stamp: 337365
<4> [514.739451] hardirqs last enabled at (337371): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.739453] hardirqs last disabled at (337376): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.739455] softirqs last enabled at (336264): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.739458] softirqs last disabled at (336257): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.739461] ---[ end trace 0000000000000000 ]---
<6> [514.739463] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.739543] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.739549] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.740655] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.740782] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.740909] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.741131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.741233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.741333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.741426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.741517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.741606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.741694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.741786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.741883] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.741985] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.742081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.743123] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.753819] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.754085] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.755128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.755217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.755301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.755383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.755463] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.755542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.755621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.755699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.755776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.755863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.755941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.756019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.756096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.756173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.756249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.756325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.756403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.756479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.756556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.756645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.756732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.756819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.756899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.756979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.757062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.757141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.757219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.757297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.757374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.757454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.757534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.757612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.757691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.757771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.757867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.757951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.758034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.758114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.758194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.758271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.758349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.758425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.758501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.758581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.758661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.758738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.758818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.758903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.758979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.759057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.759138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.759215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.759293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.759374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.759453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.759533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.759611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.759688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.759765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.759861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.759941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.760021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.760099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.760177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.760254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.760332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.760407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.760483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.760562] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.760699] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.761578] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61118, lrc_seqno=61118, guc_id=0, flags=0x73 in no process [-1]
<7> [514.761581] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.761653] ------------[ cut here ]------------
<4> [514.761654] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.761656] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:13/2860
<4> [514.761735] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.761805] [last unloaded: snd_hda_intel]
<4> [514.761809] CPU: 6 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.761815] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.761817] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.761818] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.761825] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.761901] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.761903] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.761906] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.761907] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.761908] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.761910] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.761911] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.761912] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.761914] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.761915] CR2: 00007ffdd492ffc0 CR3: 000000012f1a0001 CR4: 0000000000f72ef0
<4> [514.761917] PKRU: 55555554
<4> [514.761918] Call Trace:
<4> [514.761919] <TASK>
<4> [514.761923] ? lock_acquire+0xb0/0x300
<4> [514.761930] ? lock_release+0xd0/0x2b0
<4> [514.761935] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.761941] process_one_work+0x239/0x740
<4> [514.761949] worker_thread+0x200/0x3f0
<4> [514.761952] ? __pfx_worker_thread+0x10/0x10
<4> [514.761955] kthread+0x10d/0x150
<4> [514.761957] ? __pfx_kthread+0x10/0x10
<4> [514.761960] ret_from_fork+0x3bd/0x470
<4> [514.761963] ? __pfx_kthread+0x10/0x10
<4> [514.761965] ret_from_fork_asm+0x1a/0x30
<4> [514.761973] </TASK>
<4> [514.761974] irq event stamp: 1775473
<4> [514.761976] hardirqs last enabled at (1775479): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.761978] hardirqs last disabled at (1775484): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.761980] softirqs last enabled at (1774300): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.761983] softirqs last disabled at (1774291): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.761985] ---[ end trace 0000000000000000 ]---
<7> [514.762001] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.762110] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [514.762610] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61173, lrc_seqno=61173, guc_id=0, flags=0x73 in no process [-1]
<7> [514.762617] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.762736] ------------[ cut here ]------------
<4> [514.762738] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.762740] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#15: kworker/u64:43/4174
<4> [514.762877] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.762973] [last unloaded: snd_hda_intel]
<4> [514.762978] CPU: 15 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.762982] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.762983] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.762985] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.762994] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.763115] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.763118] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [514.763121] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.763123] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.763125] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.763127] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.763129] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.763131] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [514.763133] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.763135] CR2: 0000727e75f5b098 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [514.763137] PKRU: 55555554
<4> [514.763139] Call Trace:
<4> [514.763140] <TASK>
<4> [514.763147] ? lock_acquire+0xb0/0x300
<4> [514.763155] ? lock_release+0xd0/0x2b0
<4> [514.763163] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.763173] process_one_work+0x239/0x740
<4> [514.763184] worker_thread+0x200/0x3f0
<4> [514.763189] ? __pfx_worker_thread+0x10/0x10
<4> [514.763193] kthread+0x10d/0x150
<4> [514.763196] ? __pfx_kthread+0x10/0x10
<4> [514.763200] ret_from_fork+0x3bd/0x470
<4> [514.763203] ? __pfx_kthread+0x10/0x10
<4> [514.763208] ret_from_fork_asm+0x1a/0x30
<4> [514.763220] </TASK>
<4> [514.763222] irq event stamp: 379753
<4> [514.763223] hardirqs last enabled at (379759): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.763227] hardirqs last disabled at (379764): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.763229] softirqs last enabled at (379016): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.763233] softirqs last disabled at (379011): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.763237] ---[ end trace 0000000000000000 ]---
<6> [514.763239] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.763361] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.763369] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.763460] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.763803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.763921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.764021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.764112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.764202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.764289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.764377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.764466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.764550] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.764651] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.764746] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.765808] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.775818] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.776083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.777085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.777195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.777289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.777378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.777463] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.777546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.777629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.777712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.777795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.777893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.777975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.778057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.778138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.778219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.778300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.778380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.778462] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.778542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.778622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.778713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.778803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.778906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.778992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.779079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.779164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.779248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.779331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.779412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.779493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.779579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.779664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.779748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.779837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.779922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.780012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.780102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.780190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.780276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.780362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.780445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.780533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.780619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.780701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.780784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.780933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.781018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.781101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.781183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.781264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.781346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.781431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.781512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.781593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.781674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.781755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.781844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.781926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.782005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.782085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.782168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.782249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.782332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.782418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.782500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.782584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.782667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.782748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.782832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.782918] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.783044] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.783049] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61173, lrc_seqno=61173, guc_id=0, flags=0x73 in no process [-1]
<7> [514.783052] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.783124] ------------[ cut here ]------------
<4> [514.783125] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.783127] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [514.783211] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.783284] [last unloaded: snd_hda_intel]
<4> [514.783288] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.783291] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.783293] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.783294] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.783301] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.783382] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.783384] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.783387] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.783388] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.783390] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.783391] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.783392] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.783394] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.783395] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.783397] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [514.783398] PKRU: 55555554
<4> [514.783399] Call Trace:
<4> [514.783400] <TASK>
<4> [514.783404] ? lock_acquire+0xb0/0x300
<4> [514.783412] ? lock_release+0xd0/0x2b0
<4> [514.783416] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.783423] process_one_work+0x239/0x740
<4> [514.783431] worker_thread+0x200/0x3f0
<4> [514.783434] ? __pfx_worker_thread+0x10/0x10
<4> [514.783437] kthread+0x10d/0x150
<4> [514.783440] ? __pfx_kthread+0x10/0x10
<4> [514.783443] ret_from_fork+0x3bd/0x470
<4> [514.783446] ? __pfx_kthread+0x10/0x10
<4> [514.783449] ret_from_fork_asm+0x1a/0x30
<4> [514.783457] </TASK>
<4> [514.783458] irq event stamp: 1779929
<4> [514.783460] hardirqs last enabled at (1779935): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.783462] hardirqs last disabled at (1779940): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.783464] softirqs last enabled at (1779008): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.783468] softirqs last disabled at (1779001): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.783470] ---[ end trace 0000000000000000 ]---
<6> [514.783472] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.783553] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.783560] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.784866] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.784985] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.785096] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.785308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.785407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.785508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.785599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.785688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.785775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.785911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.786001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.786085] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.786186] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.786280] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.787319] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.797817] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.798077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.799110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.799200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.799284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.799366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.799448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.799529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.799611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.799690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.799768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.799854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.799931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.800008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.800084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.800163] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.800241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.800318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.800395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.800471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.800545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.800631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.800716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.800798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.800893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.800975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.801055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.801132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.801210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.801287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.801365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.801446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.801526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.801605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.801684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.801764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.801861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.801946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.802030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.802110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.802190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.802267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.802345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.802422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.802498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.802576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.802654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.802731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.802808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.802900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.802978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.803056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.803137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.803214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.803292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.803370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.803446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.803524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.803602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.803678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.803754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.803838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.803915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.803991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.804066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.804141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.804243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.804326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.804404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.804481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.804561] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.804684] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.804688] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61173, lrc_seqno=61173, guc_id=0, flags=0x73 in no process [-1]
<7> [514.804691] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.804759] ------------[ cut here ]------------
<4> [514.804760] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.804761] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:43/4174
<4> [514.804849] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.804918] [last unloaded: snd_hda_intel]
<4> [514.804922] CPU: 8 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.804925] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.804926] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.804928] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.804934] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.805010] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.805012] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [514.805014] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.805016] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.805017] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.805018] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.805019] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.805020] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.805022] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.805023] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [514.805025] PKRU: 55555554
<4> [514.805026] Call Trace:
<4> [514.805027] <TASK>
<4> [514.805031] ? lock_acquire+0xb0/0x300
<4> [514.805037] ? lock_release+0xd0/0x2b0
<4> [514.805042] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.805048] process_one_work+0x239/0x740
<4> [514.805055] worker_thread+0x200/0x3f0
<4> [514.805058] ? __pfx_worker_thread+0x10/0x10
<4> [514.805061] kthread+0x10d/0x150
<4> [514.805063] ? __pfx_kthread+0x10/0x10
<4> [514.805066] ret_from_fork+0x3bd/0x470
<4> [514.805069] ? __pfx_kthread+0x10/0x10
<4> [514.805072] ret_from_fork_asm+0x1a/0x30
<4> [514.805079] </TASK>
<4> [514.805081] irq event stamp: 388787
<4> [514.805082] hardirqs last enabled at (388793): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.805084] hardirqs last disabled at (388798): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.805086] softirqs last enabled at (387710): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.805089] softirqs last disabled at (387701): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.805092] ---[ end trace 0000000000000000 ]---
<7> [514.806027] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.806143] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [514.806325] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61228, lrc_seqno=61228, guc_id=0, flags=0x73 in no process [-1]
<7> [514.806328] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.806396] ------------[ cut here ]------------
<4> [514.806397] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.806398] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:43/4174
<4> [514.806475] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.806540] [last unloaded: snd_hda_intel]
<4> [514.806544] CPU: 8 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.806546] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.806547] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.806549] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.806554] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.806631] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.806633] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [514.806635] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.806636] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.806637] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.806639] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.806640] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.806641] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.806643] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.806644] CR2: 0000727e7460d098 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [514.806645] PKRU: 55555554
<4> [514.806646] Call Trace:
<4> [514.806648] <TASK>
<4> [514.806651] ? lock_acquire+0xb0/0x300
<4> [514.806656] ? lock_release+0xd0/0x2b0
<4> [514.806661] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.806667] process_one_work+0x239/0x740
<4> [514.806674] worker_thread+0x200/0x3f0
<4> [514.806677] ? __pfx_worker_thread+0x10/0x10
<4> [514.806679] kthread+0x10d/0x150
<4> [514.806682] ? __pfx_kthread+0x10/0x10
<4> [514.806685] ret_from_fork+0x3bd/0x470
<4> [514.806687] ? __pfx_kthread+0x10/0x10
<4> [514.806690] ret_from_fork_asm+0x1a/0x30
<4> [514.806698] </TASK>
<4> [514.806699] irq event stamp: 391109
<4> [514.806700] hardirqs last enabled at (391115): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.806702] hardirqs last disabled at (391120): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.806704] softirqs last enabled at (387710): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.806707] softirqs last disabled at (387701): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.806709] ---[ end trace 0000000000000000 ]---
<6> [514.806711] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.806786] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.806796] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.806928] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.807204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.807355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.807506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.807650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.807793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.807950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.808050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.808146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.808235] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.808336] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.808431] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.809468] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.819819] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.820079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.821659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.821797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.821944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.822034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.822119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.822202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.822283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.822361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.822439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.822518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.822596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.822673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.822750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.822839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.822920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.822999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.823076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.823153] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.823228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.823312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.823398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.823480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.823560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.823641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.823720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.823797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.823886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.823963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.824040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.824119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.824197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.824275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.824352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.824429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.824512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.824594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.824678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.824758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.824842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.824919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.824995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.825071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.825146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.825223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.825300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.825377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.825453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.825530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.825606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.825683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.825764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.825848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.825927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.826004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.826082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.826164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.826242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.826318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.826393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.826472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.826549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.826627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.826703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.826779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.826867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.826945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.827020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.827096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.827173] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.827420] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.827425] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61228, lrc_seqno=61228, guc_id=0, flags=0x73 in no process [-1]
<7> [514.827427] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.827497] ------------[ cut here ]------------
<4> [514.827498] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.827500] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.827579] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.827647] [last unloaded: snd_hda_intel]
<4> [514.827650] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.827653] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.827655] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.827656] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.827662] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.827739] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.827741] RSP: 0000:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.827744] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.827745] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.827747] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.827748] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.827749] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.827750] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.827752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.827753] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.827755] PKRU: 55555554
<4> [514.827756] Call Trace:
<4> [514.827757] <TASK>
<4> [514.827764] ? lock_acquire+0xb0/0x300
<4> [514.827770] ? lock_release+0xd0/0x2b0
<4> [514.827775] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.827781] process_one_work+0x239/0x740
<4> [514.827788] worker_thread+0x200/0x3f0
<4> [514.827791] ? __pfx_worker_thread+0x10/0x10
<4> [514.827794] kthread+0x10d/0x150
<4> [514.827796] ? __pfx_kthread+0x10/0x10
<4> [514.827799] ret_from_fork+0x3bd/0x470
<4> [514.827801] ? __pfx_kthread+0x10/0x10
<4> [514.827804] ret_from_fork_asm+0x1a/0x30
<4> [514.827816] </TASK>
<4> [514.827817] irq event stamp: 345933
<4> [514.827818] hardirqs last enabled at (345939): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.827821] hardirqs last disabled at (345944): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.827823] softirqs last enabled at (345068): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.827826] softirqs last disabled at (345053): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.827829] ---[ end trace 0000000000000000 ]---
<6> [514.828821] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.828905] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.828912] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.828969] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.829080] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.829405] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.829620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.829716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.829816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.829910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.829999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.830085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.830171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.830258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.830340] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.830440] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.830534] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.831572] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.841817] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.842081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.843143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.843231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.843312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.843393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.843470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.843547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.843624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.843700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.843775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.843864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.843940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.844016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.844092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.844168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.844244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.844321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.844400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.844477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.844553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.844639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.844725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.844806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.844901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.844986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.845068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.845146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.845224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.845302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.845382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.845465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.845548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.845628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.845708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.845787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.845880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.845965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.846049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.846129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.846207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.846285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.846363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.846440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.846516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.846595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.846674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.846752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.846831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.846907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.846982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.847060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.847140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.847217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.847293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.847368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.847442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.847519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.847595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.847670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.847746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.847826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.847903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.847980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.848056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.848131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.848208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.848284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.848358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.848435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.848517] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.848638] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.848642] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61228, lrc_seqno=61228, guc_id=0, flags=0x73 in no process [-1]
<7> [514.848645] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.848715] ------------[ cut here ]------------
<4> [514.848716] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.848717] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.848796] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.848868] [last unloaded: snd_hda_intel]
<4> [514.848872] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.848874] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.848876] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.848877] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.848883] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.848961] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.848963] RSP: 0000:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.848965] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.848966] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.848968] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.848969] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.848970] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.848972] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.848973] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.848975] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.848976] PKRU: 55555554
<4> [514.848977] Call Trace:
<4> [514.848979] <TASK>
<4> [514.848983] ? lock_acquire+0xb0/0x300
<4> [514.848989] ? lock_release+0xd0/0x2b0
<4> [514.848994] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.849000] process_one_work+0x239/0x740
<4> [514.849007] worker_thread+0x200/0x3f0
<4> [514.849010] ? __pfx_worker_thread+0x10/0x10
<4> [514.849013] kthread+0x10d/0x150
<4> [514.849015] ? __pfx_kthread+0x10/0x10
<4> [514.849018] ret_from_fork+0x3bd/0x470
<4> [514.849021] ? __pfx_kthread+0x10/0x10
<4> [514.849023] ret_from_fork_asm+0x1a/0x30
<4> [514.849031] </TASK>
<4> [514.849032] irq event stamp: 349097
<4> [514.849033] hardirqs last enabled at (349103): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.849036] hardirqs last disabled at (349108): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.849037] softirqs last enabled at (348020): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.849040] softirqs last disabled at (348013): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.849043] ---[ end trace 0000000000000000 ]---
<7> [514.850327] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.850445] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [514.850550] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61283, lrc_seqno=61283, guc_id=0, flags=0x73 in no process [-1]
<7> [514.850553] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.850622] ------------[ cut here ]------------
<4> [514.850623] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.850624] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.850704] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.850768] [last unloaded: snd_hda_intel]
<4> [514.850771] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.850774] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.850775] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.850776] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.850782] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.850869] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.850871] RSP: 0000:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.850874] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.850875] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.850877] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.850878] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.850879] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.850880] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.850882] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.850883] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.850885] PKRU: 55555554
<4> [514.850886] Call Trace:
<4> [514.850887] <TASK>
<4> [514.850891] ? lock_acquire+0xb0/0x300
<4> [514.850896] ? lock_release+0xd0/0x2b0
<4> [514.850901] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.850907] process_one_work+0x239/0x740
<4> [514.850914] worker_thread+0x200/0x3f0
<4> [514.850917] ? __pfx_worker_thread+0x10/0x10
<4> [514.850920] kthread+0x10d/0x150
<4> [514.850922] ? __pfx_kthread+0x10/0x10
<4> [514.850925] ret_from_fork+0x3bd/0x470
<4> [514.850928] ? __pfx_kthread+0x10/0x10
<4> [514.850931] ret_from_fork_asm+0x1a/0x30
<4> [514.850938] </TASK>
<4> [514.850939] irq event stamp: 351433
<4> [514.850941] hardirqs last enabled at (351439): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.850943] hardirqs last disabled at (351444): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.850945] softirqs last enabled at (351272): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.850948] softirqs last disabled at (351261): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.850950] ---[ end trace 0000000000000000 ]---
<6> [514.850952] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.851029] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.851037] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.851225] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.851508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.851652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.851799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.851970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.852081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.852180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.852274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.852366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.852452] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.852553] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.852649] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.853689] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.864420] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.864679] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.865731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.865819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.865901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.865982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.866063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.866142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.866223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.866303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.866381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.866457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.866532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.866609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.866685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.866761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.866844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.866921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.866997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.867073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.867149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.867235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.867319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.867400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.867480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.867560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.867639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.867717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.867794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.867883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.867960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.868041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.868120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.868198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.868277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.868355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.868441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.868525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.868610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.868691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.868771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.868857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.868936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.869014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.869091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.869169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.869248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.869326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.869404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.869482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.869558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.869637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.869718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.869796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.869887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.869968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.870047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.870130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.870210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.870286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.870363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.870443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.870522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.870597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.870672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.870748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.870825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.870903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.870979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.871053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.871132] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.871255] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.871259] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61283, lrc_seqno=61283, guc_id=0, flags=0x73 in no process [-1]
<7> [514.871261] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.871331] ------------[ cut here ]------------
<4> [514.871332] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.871333] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [514.871411] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.871477] [last unloaded: snd_hda_intel]
<4> [514.871481] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.871483] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.871485] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.871486] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.871492] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.871569] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.871571] RSP: 0000:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.871573] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.871574] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.871576] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.871577] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.871578] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.871579] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.871581] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.871582] CR2: 0000727e7460d098 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [514.871584] PKRU: 55555554
<4> [514.871585] Call Trace:
<4> [514.871586] <TASK>
<4> [514.871590] ? lock_acquire+0xb0/0x300
<4> [514.871596] ? lock_release+0xd0/0x2b0
<4> [514.871601] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.871607] process_one_work+0x239/0x740
<4> [514.871613] worker_thread+0x200/0x3f0
<4> [514.871617] ? __pfx_worker_thread+0x10/0x10
<4> [514.871619] kthread+0x10d/0x150
<4> [514.871622] ? __pfx_kthread+0x10/0x10
<4> [514.871625] ret_from_fork+0x3bd/0x470
<4> [514.871627] ? __pfx_kthread+0x10/0x10
<4> [514.871630] ret_from_fork_asm+0x1a/0x30
<4> [514.871637] </TASK>
<4> [514.871638] irq event stamp: 1793669
<4> [514.871640] hardirqs last enabled at (1793675): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.871642] hardirqs last disabled at (1793680): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.871644] softirqs last enabled at (1792542): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.871647] softirqs last disabled at (1792537): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.871649] ---[ end trace 0000000000000000 ]---
<6> [514.871651] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.872549] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.872556] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.872639] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.872755] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.872875] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.873096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.873197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.873297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.873391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.873481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.873571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.873660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.873750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.873841] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.873945] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.874042] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.875117] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.885817] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.886079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.887109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.887196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.887279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.887362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.887446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.887529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.887610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.887689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.887766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.887852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.887929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.888005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.888081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.888159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.888235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.888310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.888385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.888460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.888535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.888621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.888705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.888787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.888875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.888957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.889038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.889117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.889195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.889276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.889356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.889436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.889517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.889597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.889677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.889757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.889847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.889931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.890015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.890097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.890177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.890255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.890333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.890410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.890486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.890563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.890642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.890720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.890797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.890888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.890981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.891061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.891142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.891220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.891298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.891376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.891453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.891531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.891607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.891683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.891758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.891864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.891942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.892019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.892095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.892170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.892247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.892324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.892399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.892474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.892552] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.892675] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.892679] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61283, lrc_seqno=61283, guc_id=0, flags=0x73 in no process [-1]
<7> [514.892682] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.892750] ------------[ cut here ]------------
<4> [514.892751] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.892753] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.892837] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.892906] [last unloaded: snd_hda_intel]
<4> [514.892910] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.892913] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.892914] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.892915] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.892921] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.892997] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.892999] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.893001] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.893003] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.893004] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.893005] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.893007] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.893008] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.893010] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.893011] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.893012] PKRU: 55555554
<4> [514.893014] Call Trace:
<4> [514.893015] <TASK>
<4> [514.893019] ? lock_acquire+0xb0/0x300
<4> [514.893024] ? lock_release+0xd0/0x2b0
<4> [514.893029] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.893035] process_one_work+0x239/0x740
<4> [514.893042] worker_thread+0x200/0x3f0
<4> [514.893045] ? __pfx_worker_thread+0x10/0x10
<4> [514.893048] kthread+0x10d/0x150
<4> [514.893050] ? __pfx_kthread+0x10/0x10
<4> [514.893053] ret_from_fork+0x3bd/0x470
<4> [514.893056] ? __pfx_kthread+0x10/0x10
<4> [514.893059] ret_from_fork_asm+0x1a/0x30
<4> [514.893066] </TASK>
<4> [514.893068] irq event stamp: 360421
<4> [514.893069] hardirqs last enabled at (360427): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.893071] hardirqs last disabled at (360432): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.893073] softirqs last enabled at (359524): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.893076] softirqs last disabled at (359383): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.893079] ---[ end trace 0000000000000000 ]---
<5> [514.893462] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61338, lrc_seqno=61338, guc_id=0, flags=0x73 in no process [-1]
<7> [514.893469] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.893977] ------------[ cut here ]------------
<4> [514.893980] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.893982] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#12: kworker/u64:43/4174
<4> [514.894115] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.894219] [last unloaded: snd_hda_intel]
<4> [514.894225] CPU: 12 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.894229] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.894230] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.894232] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.894244] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.894386] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.894389] RSP: 0000:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [514.894393] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.894395] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.894397] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.894399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.894401] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.894403] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [514.894405] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.894407] CR2: 0000727e77d76098 CR3: 000000000364a003 CR4: 0000000000f72ef0
<4> [514.894410] PKRU: 55555554
<4> [514.894411] Call Trace:
<4> [514.894413] <TASK>
<4> [514.894420] ? lock_acquire+0xb0/0x300
<4> [514.894428] ? lock_release+0xd0/0x2b0
<4> [514.894436] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.894446] process_one_work+0x239/0x740
<4> [514.894457] worker_thread+0x200/0x3f0
<4> [514.894461] ? __pfx_worker_thread+0x10/0x10
<4> [514.894465] kthread+0x10d/0x150
<4> [514.894469] ? __pfx_kthread+0x10/0x10
<4> [514.894473] ret_from_fork+0x3bd/0x470
<4> [514.894477] ? __pfx_kthread+0x10/0x10
<4> [514.894481] ret_from_fork_asm+0x1a/0x30
<4> [514.894494] </TASK>
<4> [514.894495] irq event stamp: 393707
<4> [514.894497] hardirqs last enabled at (393713): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.894501] hardirqs last disabled at (393718): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.894504] softirqs last enabled at (392950): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.894508] softirqs last disabled at (392945): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.894513] ---[ end trace 0000000000000000 ]---
<6> [514.894529] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.894649] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.894658] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.894715] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.894841] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.895339] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.895555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.895652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.895750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.895850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.895939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.896025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.896112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.896199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.896281] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.896382] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.896476] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.897512] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.907816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.908076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.909160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.909245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.909327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.909408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.909489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.909568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.909646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.909723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.909800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.909931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.910012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.910090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.910166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.910242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.910317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.910396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.910473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.910549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.910625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.910710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.910795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.910890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.910975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.911056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.911136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.911212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.911288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.911365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.911441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.911520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.911599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.911677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.911755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.911838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.911921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.912005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.912087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.912166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.912245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.912322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.912398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.912475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.912551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.912629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.912708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.912784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.912868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.912943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.913019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.913097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.913176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.913253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.913330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.913407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.913481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.913558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.913632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.913707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.913782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.913870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.913952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.914032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.914110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.914187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.914265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.914342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.914417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.914493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.914570] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.914689] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.914694] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61338, lrc_seqno=61338, guc_id=0, flags=0x73 in no process [-1]
<7> [514.914696] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.914766] ------------[ cut here ]------------
<4> [514.914768] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.914769] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.914871] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.914942] [last unloaded: snd_hda_intel]
<4> [514.914946] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.914949] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.914951] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.914952] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.914958] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.915036] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.915038] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.915040] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.915041] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.915043] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.915044] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.915045] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.915047] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.915048] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.915050] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.915051] PKRU: 55555554
<4> [514.915052] Call Trace:
<4> [514.915053] <TASK>
<4> [514.915058] ? lock_acquire+0xb0/0x300
<4> [514.915064] ? lock_release+0xd0/0x2b0
<4> [514.915069] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.915075] process_one_work+0x239/0x740
<4> [514.915083] worker_thread+0x200/0x3f0
<4> [514.915086] ? __pfx_worker_thread+0x10/0x10
<4> [514.915089] kthread+0x10d/0x150
<4> [514.915091] ? __pfx_kthread+0x10/0x10
<4> [514.915094] ret_from_fork+0x3bd/0x470
<4> [514.915097] ? __pfx_kthread+0x10/0x10
<4> [514.915100] ret_from_fork_asm+0x1a/0x30
<4> [514.915108] </TASK>
<4> [514.915110] irq event stamp: 364971
<4> [514.915111] hardirqs last enabled at (364977): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.915114] hardirqs last disabled at (364982): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.915115] softirqs last enabled at (363806): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.915119] softirqs last disabled at (363799): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.915121] ---[ end trace 0000000000000000 ]---
<6> [514.915123] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.915200] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.915205] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.915430] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.915639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.915730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.915832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.915922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.916009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.916094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.916179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.916266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.916348] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.916448] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.916542] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.917576] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.927816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.928079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.929096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.929184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.929264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.929345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.929424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.929502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.929580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.929657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.929734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.929816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.929895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.929973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.930049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.930125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.930199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.930274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.930349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.930424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.930498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.930582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.930665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.930747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.930829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.930911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.930990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.931068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.931145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.931222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.931300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.931378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.931457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.931535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.931615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.931694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.931778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.931870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.931955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.932037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.932117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.932193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.932270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.932346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.932422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.932499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.932576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.932654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.932732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.932815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.932894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.932973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.933054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.933132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.933209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.933286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.933361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.933438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.933519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.933597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.933675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.933754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.933836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.933914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.933991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.934068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.934146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.934223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.934299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.934375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.934455] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.934576] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.934580] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61338, lrc_seqno=61338, guc_id=0, flags=0x73 in no process [-1]
<7> [514.934583] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.934651] ------------[ cut here ]------------
<4> [514.934652] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.934653] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.934743] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.934812] [last unloaded: snd_hda_intel]
<4> [514.934815] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.934818] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.934819] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.934821] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.934827] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.934903] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.934905] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.934907] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.934909] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.934910] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.934911] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.934912] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.934914] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.934915] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.934916] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.934918] PKRU: 55555554
<4> [514.934919] Call Trace:
<4> [514.934920] <TASK>
<4> [514.934924] ? lock_acquire+0xb0/0x300
<4> [514.934929] ? lock_release+0xd0/0x2b0
<4> [514.934934] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.934940] process_one_work+0x239/0x740
<4> [514.934947] worker_thread+0x200/0x3f0
<4> [514.934950] ? __pfx_worker_thread+0x10/0x10
<4> [514.934953] kthread+0x10d/0x150
<4> [514.934956] ? __pfx_kthread+0x10/0x10
<4> [514.934959] ret_from_fork+0x3bd/0x470
<4> [514.934961] ? __pfx_kthread+0x10/0x10
<4> [514.934964] ret_from_fork_asm+0x1a/0x30
<4> [514.934971] </TASK>
<4> [514.934973] irq event stamp: 368123
<4> [514.934974] hardirqs last enabled at (368129): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.934976] hardirqs last disabled at (368134): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.934978] softirqs last enabled at (366962): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.934981] softirqs last disabled at (366955): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.934984] ---[ end trace 0000000000000000 ]---
<7> [514.936165] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.936281] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [514.936385] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61393, lrc_seqno=61393, guc_id=0, flags=0x73 in no process [-1]
<7> [514.936388] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.936456] ------------[ cut here ]------------
<4> [514.936458] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.936459] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.936538] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.936602] [last unloaded: snd_hda_intel]
<4> [514.936605] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.936608] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.936609] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.936610] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.936616] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.936692] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.936694] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.936696] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.936697] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.936699] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.936700] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.936701] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.936702] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.936704] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.936705] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.936706] PKRU: 55555554
<4> [514.936708] Call Trace:
<4> [514.936709] <TASK>
<4> [514.936712] ? lock_acquire+0xb0/0x300
<4> [514.936718] ? lock_release+0xd0/0x2b0
<4> [514.936722] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.936729] process_one_work+0x239/0x740
<4> [514.936735] worker_thread+0x200/0x3f0
<4> [514.936739] ? __pfx_worker_thread+0x10/0x10
<4> [514.936741] kthread+0x10d/0x150
<4> [514.936744] ? __pfx_kthread+0x10/0x10
<4> [514.936747] ret_from_fork+0x3bd/0x470
<4> [514.936749] ? __pfx_kthread+0x10/0x10
<4> [514.936752] ret_from_fork_asm+0x1a/0x30
<4> [514.936759] </TASK>
<4> [514.936761] irq event stamp: 370087
<4> [514.936762] hardirqs last enabled at (370093): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.936764] hardirqs last disabled at (370098): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.936766] softirqs last enabled at (366962): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.936769] softirqs last disabled at (366955): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.936771] ---[ end trace 0000000000000000 ]---
<6> [514.936773] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.936860] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.936865] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.936997] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.937208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.937302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.937398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.937503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.937592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.937681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.937769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.937869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.937955] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.938054] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.938148] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.939195] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.949816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.950077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.951441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.951534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.951617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.951699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.951779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.951868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.951966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.952044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.952122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.952199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.952277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.952357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.952436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.952513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.952589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.952664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.952738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.952816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.952891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.952976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.953061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.953142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.953221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.953301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.953382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.953463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.953541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.953618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.953694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.953774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.953862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.953941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.954020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.954099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.954183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.954269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.954355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.954436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.954517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.954597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.954677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.954755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.954838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.954921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.955003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.955081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.955159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.955237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.955312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.955388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.955466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.955543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.955620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.955695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.955771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.955857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.955934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.956010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.956086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.956164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.956241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.956319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.956396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.956471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.956549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.956625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.956700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.956775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.956860] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.956980] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.956984] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61393, lrc_seqno=61393, guc_id=0, flags=0x73 in no process [-1]
<7> [514.956986] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.957054] ------------[ cut here ]------------
<4> [514.957055] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.957056] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:45/4176
<4> [514.957134] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.957199] [last unloaded: snd_hda_intel]
<4> [514.957202] CPU: 6 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.957205] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.957206] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.957208] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.957213] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.957290] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.957291] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [514.957294] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.957295] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.957296] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.957297] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.957299] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.957300] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [514.957302] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.957303] CR2: 00007ffdd492ffc0 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [514.957304] PKRU: 55555554
<4> [514.957306] Call Trace:
<4> [514.957307] <TASK>
<4> [514.957311] ? lock_acquire+0xb0/0x300
<4> [514.957316] ? lock_release+0xd0/0x2b0
<4> [514.957321] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.957327] process_one_work+0x239/0x740
<4> [514.957334] worker_thread+0x200/0x3f0
<4> [514.957337] ? __pfx_worker_thread+0x10/0x10
<4> [514.957340] kthread+0x10d/0x150
<4> [514.957342] ? __pfx_kthread+0x10/0x10
<4> [514.957345] ret_from_fork+0x3bd/0x470
<4> [514.957348] ? __pfx_kthread+0x10/0x10
<4> [514.957350] ret_from_fork_asm+0x1a/0x30
<4> [514.957358] </TASK>
<4> [514.957359] irq event stamp: 373247
<4> [514.957360] hardirqs last enabled at (373253): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.957363] hardirqs last disabled at (373258): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.957364] softirqs last enabled at (372410): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.957367] softirqs last disabled at (372405): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.957370] ---[ end trace 0000000000000000 ]---
<6> [514.957372] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.957449] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.957455] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.958360] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [514.958477] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [514.958588] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.958864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.958967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.959066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.959158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.959246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.959333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.959421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.959511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.959599] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.959702] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.959797] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.960875] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.970818] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.971079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.972081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.972166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.972247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.972329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.972408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.972486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.972565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.972643] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.972720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.972797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.972887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.972964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.973041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.973116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.973192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.973268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.973343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.973418] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.973494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.973581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.973665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.973746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.973830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.973910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.973992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.974069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.974145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.974221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.974296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.974374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.974454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.974532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.974610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.974687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.974769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.974863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.974947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.975026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.975105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.975181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.975257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.975333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.975408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.975485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.975561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.975638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.975715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.975794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.975891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.975971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.976051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.976127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.976203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.976280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.976356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.976433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.976512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.976591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.976668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.976747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.976828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.976907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.976984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.977060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.977139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.977217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.977292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.977368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.977447] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.977569] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.977574] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61393, lrc_seqno=61393, guc_id=0, flags=0x73 in no process [-1]
<7> [514.977577] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.977645] ------------[ cut here ]------------
<4> [514.977647] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.977648] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [514.977737] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.977806] [last unloaded: snd_hda_intel]
<4> [514.978176] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.978179] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.978180] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.978182] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.978188] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.978266] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.978268] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.978271] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.978272] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.978273] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.978275] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.978276] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.978277] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.978279] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.978280] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [514.978282] PKRU: 55555554
<4> [514.978283] Call Trace:
<4> [514.978284] <TASK>
<4> [514.978288] ? lock_acquire+0xb0/0x300
<4> [514.978294] ? lock_release+0xd0/0x2b0
<4> [514.978299] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.978306] process_one_work+0x239/0x740
<4> [514.978313] worker_thread+0x200/0x3f0
<4> [514.978316] ? __pfx_worker_thread+0x10/0x10
<4> [514.978319] kthread+0x10d/0x150
<4> [514.978321] ? __pfx_kthread+0x10/0x10
<4> [514.978324] ret_from_fork+0x3bd/0x470
<4> [514.978327] ? __pfx_kthread+0x10/0x10
<4> [514.978329] ret_from_fork_asm+0x1a/0x30
<4> [514.978337] </TASK>
<4> [514.978339] irq event stamp: 1813929
<4> [514.978340] hardirqs last enabled at (1813935): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.978342] hardirqs last disabled at (1813940): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.978344] softirqs last enabled at (1813038): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.978347] softirqs last disabled at (1813031): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [514.978350] ---[ end trace 0000000000000000 ]---
<7> [514.979277] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<5> [514.979381] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61448, lrc_seqno=61448, guc_id=0, flags=0x73 in no process [-1]
<7> [514.979387] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [514.979389] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [514.979465] ------------[ cut here ]------------
<4> [514.979466] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.979467] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:43/4174
<4> [514.979554] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.979620] [last unloaded: snd_hda_intel]
<4> [514.979624] CPU: 1 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.979627] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.979628] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.979630] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.979635] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.979715] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.979717] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [514.979719] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.979721] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.979722] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.979723] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.979725] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.979726] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [514.979728] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.979729] CR2: 0000727e7e545c60 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [514.979731] PKRU: 55555554
<4> [514.979732] Call Trace:
<4> [514.979733] <TASK>
<4> [514.979737] ? lock_acquire+0xb0/0x300
<4> [514.979743] ? lock_release+0xd0/0x2b0
<4> [514.979748] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [514.979754] process_one_work+0x239/0x740
<4> [514.979761] worker_thread+0x200/0x3f0
<4> [514.979764] ? __pfx_worker_thread+0x10/0x10
<4> [514.979767] kthread+0x10d/0x150
<4> [514.979769] ? __pfx_kthread+0x10/0x10
<4> [514.979772] ret_from_fork+0x3bd/0x470
<4> [514.979774] ? __pfx_kthread+0x10/0x10
<4> [514.979777] ret_from_fork_asm+0x1a/0x30
<4> [514.979785] </TASK>
<4> [514.979786] irq event stamp: 399963
<4> [514.979787] hardirqs last enabled at (399969): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [514.979789] hardirqs last disabled at (399974): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [514.979791] softirqs last enabled at (398618): [<ffffffff8133bdb3>] kernel_fpu_end+0x53/0x70
<4> [514.979794] softirqs last disabled at (398616): [<ffffffff8133c4c4>] kernel_fpu_begin_mask+0xc4/0x120
<4> [514.979796] ---[ end trace 0000000000000000 ]---
<6> [514.979799] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [514.979896] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [514.979905] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [514.980110] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [514.980325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [514.980421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [514.980518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [514.980608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [514.980696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [514.980783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [514.980879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [514.980968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [514.981051] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [514.981151] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [514.981245] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [514.982297] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [514.992815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [514.993077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [514.994113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [514.994201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [514.994282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [514.994361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [514.994439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [514.994517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [514.994594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [514.994670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [514.994747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [514.994826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [514.994918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [514.994994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [514.995069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [514.995146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [514.995221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [514.995297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [514.995373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [514.995448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [514.995524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [514.995609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [514.995695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [514.995775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [514.995866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [514.995949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [514.996029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [514.996108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [514.996188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [514.996269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [514.996348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [514.996427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [514.996506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [514.996584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [514.996663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [514.996741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [514.996827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [514.996912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [514.996996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [514.997075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [514.997155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [514.997232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [514.997309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [514.997386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [514.997462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [514.997539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [514.997621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [514.997702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [514.997781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [514.997866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [514.997943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [514.998023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [514.998104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [514.998181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [514.998259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [514.998336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [514.998412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [514.998490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [514.998567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [514.998644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [514.998720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [514.998799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [514.998909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [514.998987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [514.999063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [514.999139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [514.999221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [514.999300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [514.999377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [514.999453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [514.999531] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [514.999651] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [514.999655] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61448, lrc_seqno=61448, guc_id=0, flags=0x73 in no process [-1]
<7> [514.999657] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [514.999725] ------------[ cut here ]------------
<4> [514.999726] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [514.999727] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [514.999807] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [514.999878] [last unloaded: snd_hda_intel]
<4> [514.999882] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [514.999885] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [514.999886] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [514.999888] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [514.999893] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [514.999971] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [514.999973] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [514.999975] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [514.999977] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [514.999978] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [514.999980] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [514.999981] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [514.999982] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [514.999984] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [514.999986] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [514.999988] PKRU: 55555554
<4> [514.999989] Call Trace:
<4> [514.999990] <TASK>
<4> [514.999995] ? lock_acquire+0xb0/0x300
<4> [515.000000] ? lock_release+0xd0/0x2b0
<4> [515.000005] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.000011] process_one_work+0x239/0x740
<4> [515.000018] worker_thread+0x200/0x3f0
<4> [515.000022] ? __pfx_worker_thread+0x10/0x10
<4> [515.000025] kthread+0x10d/0x150
<4> [515.000027] ? __pfx_kthread+0x10/0x10
<4> [515.000030] ret_from_fork+0x3bd/0x470
<4> [515.000033] ? __pfx_kthread+0x10/0x10
<4> [515.000036] ret_from_fork_asm+0x1a/0x30
<4> [515.000043] </TASK>
<4> [515.000045] irq event stamp: 1817717
<4> [515.000046] hardirqs last enabled at (1817723): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.000048] hardirqs last disabled at (1817728): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.000050] softirqs last enabled at (1816828): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.000053] softirqs last disabled at (1816817): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.000056] ---[ end trace 0000000000000000 ]---
<6> [515.000058] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.000138] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.000145] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.000404] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.000614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.000711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.000806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.000911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.001000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.001088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.001175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.001264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.001348] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.001449] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.001544] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.002580] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.013165] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.013428] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.014486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.014575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.014657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.014737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.014821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.014899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.014977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.015054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.015129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.015206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.015282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.015357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.015434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.015509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.015585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.015659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.015734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.015814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.015895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.015984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.016068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.016150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.016230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.016311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.016391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.016469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.016546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.016623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.016700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.016780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.016871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.016951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.017030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.017110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.017195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.017279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.017364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.017445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.017543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.017622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.017701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.017781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.017866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.017944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.018024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.018102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.018179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.018256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.018331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.018409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.018488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.018564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.018642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.018720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.018795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.018884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.018964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.019043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.019120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.019201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.019280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.019358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.019435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.019511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.019589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.019666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.019742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.019819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.019918] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.020041] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.020045] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61448, lrc_seqno=61448, guc_id=0, flags=0x73 in no process [-1]
<7> [515.020047] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.020115] ------------[ cut here ]------------
<4> [515.020116] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.020117] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [515.020196] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.020260] [last unloaded: snd_hda_intel]
<4> [515.020263] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.020266] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.020267] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.020268] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.020274] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.020352] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.020353] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.020356] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.020357] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.020358] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.020360] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.020361] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.020362] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.020364] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.020365] CR2: 0000727e7460d098 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [515.020367] PKRU: 55555554
<4> [515.020368] Call Trace:
<4> [515.020369] <TASK>
<4> [515.020373] ? lock_acquire+0xb0/0x300
<4> [515.020378] ? lock_release+0xd0/0x2b0
<4> [515.020383] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.020389] process_one_work+0x239/0x740
<4> [515.020396] worker_thread+0x200/0x3f0
<4> [515.020399] ? __pfx_worker_thread+0x10/0x10
<4> [515.020401] kthread+0x10d/0x150
<4> [515.020404] ? __pfx_kthread+0x10/0x10
<4> [515.020407] ret_from_fork+0x3bd/0x470
<4> [515.020409] ? __pfx_kthread+0x10/0x10
<4> [515.020412] ret_from_fork_asm+0x1a/0x30
<4> [515.020419] </TASK>
<4> [515.020420] irq event stamp: 1820873
<4> [515.020421] hardirqs last enabled at (1820879): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.020424] hardirqs last disabled at (1820884): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.020425] softirqs last enabled at (1819864): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.020428] softirqs last disabled at (1819857): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.020431] ---[ end trace 0000000000000000 ]---
<5> [515.020908] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61503, lrc_seqno=61503, guc_id=0, flags=0x73 in no process [-1]
<7> [515.020918] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.021281] ------------[ cut here ]------------
<4> [515.021283] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.021286] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#9: kworker/u64:13/2860
<4> [515.021563] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.021628] [last unloaded: snd_hda_intel]
<4> [515.021631] CPU: 9 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.021635] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.021637] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.021639] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.021648] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<7> [515.021656] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [515.021775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.021777] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.021781] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.021783] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.021784] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.021786] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.021788] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.021790] FS: 0000000000000000(0000) GS:ffff8888daf03000(0000) knlGS:0000000000000000
<4> [515.021792] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.021794] CR2: 0000727e7401b098 CR3: 000000000364a002 CR4: 0000000000f72ef0
<4> [515.021796] PKRU: 55555554
<4> [515.021798] Call Trace:
<4> [515.021799] <TASK>
<4> [515.021817] ? lock_acquire+0xb0/0x300
<4> [515.021830] ? lock_release+0xd0/0x2b0
<4> [515.021838] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.021849] process_one_work+0x239/0x740
<4> [515.021859] worker_thread+0x200/0x3f0
<4> [515.021863] ? __pfx_worker_thread+0x10/0x10
<4> [515.021868] kthread+0x10d/0x150
<4> [515.021871] ? __pfx_kthread+0x10/0x10
<4> [515.021876] ret_from_fork+0x3bd/0x470
<4> [515.021880] ? __pfx_kthread+0x10/0x10
<4> [515.021885] ret_from_fork_asm+0x1a/0x30
<4> [515.021898] </TASK>
<4> [515.021900] irq event stamp: 1822339
<7> [515.021827] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [515.021902] hardirqs last enabled at (1822345): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.021905] hardirqs last disabled at (1822350): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.021908] softirqs last enabled at (1822266): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.021912] softirqs last disabled at (1822261): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.021915] ---[ end trace 0000000000000000 ]---
<6> [515.021918] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.022023] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.022031] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.022215] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.022431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.022528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.022625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.022718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.022816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.022907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.022994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.023082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.023165] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.023266] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.023360] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.024402] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.035133] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.035394] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.036415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.036505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.036585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.036665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.036743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.036824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.036901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.036977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.037054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.037130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.037206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.037281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.037359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.037435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.037510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.037584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.037659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.037734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.038265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.038357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.038443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.038527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.038610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.038691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.038770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.038857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.038934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.039011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.039088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.039166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.039245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.039324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.039403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.039481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.039565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.039649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.039733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.039817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.039898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.039973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.040049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.040126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.040201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.040279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.040358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.040436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.040514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.040590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.040666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.040741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.040826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.040903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.040981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.041057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.041131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.041208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.041285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.041360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.041435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.041513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.041588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.041664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.041741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.041819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.041895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.041969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.042044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.042119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.042196] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.042319] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.042323] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61503, lrc_seqno=61503, guc_id=0, flags=0x73 in no process [-1]
<7> [515.042326] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.042394] ------------[ cut here ]------------
<4> [515.042395] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.042397] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#9: kworker/u64:13/2860
<4> [515.042475] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.042540] [last unloaded: snd_hda_intel]
<4> [515.042543] CPU: 9 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.042546] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.042547] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.042549] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.042554] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.042633] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.042635] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.042637] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.042638] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.042640] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.042641] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.042642] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.042644] FS: 0000000000000000(0000) GS:ffff8888daf03000(0000) knlGS:0000000000000000
<4> [515.042645] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.042646] CR2: 0000727e7401b098 CR3: 000000000364a002 CR4: 0000000000f72ef0
<4> [515.042648] PKRU: 55555554
<4> [515.042649] Call Trace:
<4> [515.042650] <TASK>
<4> [515.042654] ? lock_acquire+0xb0/0x300
<4> [515.042660] ? lock_release+0xd0/0x2b0
<4> [515.042665] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.042671] process_one_work+0x239/0x740
<4> [515.042678] worker_thread+0x200/0x3f0
<4> [515.042681] ? __pfx_worker_thread+0x10/0x10
<4> [515.042684] kthread+0x10d/0x150
<4> [515.042686] ? __pfx_kthread+0x10/0x10
<4> [515.042689] ret_from_fork+0x3bd/0x470
<4> [515.042691] ? __pfx_kthread+0x10/0x10
<4> [515.042694] ret_from_fork_asm+0x1a/0x30
<4> [515.042701] </TASK>
<4> [515.042703] irq event stamp: 1825511
<4> [515.042704] hardirqs last enabled at (1825517): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.042706] hardirqs last disabled at (1825522): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.042708] softirqs last enabled at (1824642): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.042711] softirqs last disabled at (1824635): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.042713] ---[ end trace 0000000000000000 ]---
<6> [515.042829] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.043225] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.043237] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.043625] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.043739] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.044113] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.044330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.044429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.044528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.044621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.044713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.044802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.044917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.045011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.045096] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.045199] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.045295] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.046339] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.056832] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.057095] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.058168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.058256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.058338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.058419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.058498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.058577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.058654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.058732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.059127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.059207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.059284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.059360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.059437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.059514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.059591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.059667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.059743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.059823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.059901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.059988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.060073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.060155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.060235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.060317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.060397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.060475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.060556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.060637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.060715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.060795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.060889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.060968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.061047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.061127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.061211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.061294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.061378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.061459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.061539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.061616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.061694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.061770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.061855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.061932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.062011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.062088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.062166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.062243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.062318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.062396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.062476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.062552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.062629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.062705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.062781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.062866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.062943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.063018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.063093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.063171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.063248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.063325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.063407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.063485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.063563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.063641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.063718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.063797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.063894] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.064016] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.064021] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61503, lrc_seqno=61503, guc_id=0, flags=0x73 in no process [-1]
<7> [515.064023] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.064092] ------------[ cut here ]------------
<4> [515.064093] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.064095] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [515.064174] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.064242] [last unloaded: snd_hda_intel]
<4> [515.064245] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.064248] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.064250] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.064251] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.064257] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.064333] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.064335] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.064337] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.064339] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.064340] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.064342] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.064343] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.064344] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.064346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.064347] CR2: 0000727e7460d098 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [515.064349] PKRU: 55555554
<4> [515.064350] Call Trace:
<4> [515.064351] <TASK>
<4> [515.064355] ? lock_acquire+0xb0/0x300
<4> [515.064361] ? lock_release+0xd0/0x2b0
<4> [515.064366] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.064372] process_one_work+0x239/0x740
<4> [515.064379] worker_thread+0x200/0x3f0
<4> [515.064382] ? __pfx_worker_thread+0x10/0x10
<4> [515.064384] kthread+0x10d/0x150
<4> [515.064387] ? __pfx_kthread+0x10/0x10
<4> [515.064390] ret_from_fork+0x3bd/0x470
<4> [515.064392] ? __pfx_kthread+0x10/0x10
<4> [515.064395] ret_from_fork_asm+0x1a/0x30
<4> [515.064403] </TASK>
<4> [515.064404] irq event stamp: 388355
<4> [515.064405] hardirqs last enabled at (388361): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.064408] hardirqs last disabled at (388366): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.064409] softirqs last enabled at (387518): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.064412] softirqs last disabled at (387505): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.064415] ---[ end trace 0000000000000000 ]---
<5> [515.065811] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61558, lrc_seqno=61558, guc_id=0, flags=0x73 in no process [-1]
<7> [515.065815] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.065891] ------------[ cut here ]------------
<4> [515.065892] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.065894] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [515.065977] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.066042] [last unloaded: snd_hda_intel]
<4> [515.066045] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.066047] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.066049] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.066050] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.066056] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.066134] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.066136] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.066138] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.066140] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.066141] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.066142] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.066143] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.066145] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.066146] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.066147] CR2: 0000727e7460d098 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [515.066149] PKRU: 55555554
<4> [515.066150] Call Trace:
<4> [515.066151] <TASK>
<4> [515.066155] ? lock_acquire+0xb0/0x300
<4> [515.066160] ? lock_release+0xd0/0x2b0
<4> [515.066165] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.066171] process_one_work+0x239/0x740
<4> [515.066177] worker_thread+0x200/0x3f0
<4> [515.066181] ? __pfx_worker_thread+0x10/0x10
<4> [515.066183] kthread+0x10d/0x150
<4> [515.066186] ? __pfx_kthread+0x10/0x10
<4> [515.066189] ret_from_fork+0x3bd/0x470
<4> [515.066191] ? __pfx_kthread+0x10/0x10
<4> [515.066194] ret_from_fork_asm+0x1a/0x30
<4> [515.066201] </TASK>
<4> [515.066202] irq event stamp: 389797
<4> [515.066204] hardirqs last enabled at (389803): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.066206] hardirqs last disabled at (389808): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.066208] softirqs last enabled at (389020): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.066211] softirqs last disabled at (389013): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.066213] ---[ end trace 0000000000000000 ]---
<6> [515.066215] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.066292] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.066299] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.066316] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.066428] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.066541] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.066765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.066882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.066987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.067083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.067174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.067262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.067349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.067437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.067520] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.067619] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.067715] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.068754] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.078822] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.079091] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.080164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.080255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.080337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.080418] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.080496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.080573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.080651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.080731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.080857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.080936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.081014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.081090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.081165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.081240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.081316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.081391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.081467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.081544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.081618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.081706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.081793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.081887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.081966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.082046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.082124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.082201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.082278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.082355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.082431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.082510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.082589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.082668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.082746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.082828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.082913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.082997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.083081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.083163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.083243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.083320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.083398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.083474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.083549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.083626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.083705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.083783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.083872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.083971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.084048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.084125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.084205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.084282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.084360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.084437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.084513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.084589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.084666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.084742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.084823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.084902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.084978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.085056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.085132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.085207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.085283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.085359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.085432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.085506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.085584] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.085705] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.085710] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61558, lrc_seqno=61558, guc_id=0, flags=0x73 in no process [-1]
<7> [515.085713] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.085781] ------------[ cut here ]------------
<4> [515.085783] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.085784] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:13/2860
<4> [515.085871] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.085939] [last unloaded: snd_hda_intel]
<4> [515.085942] CPU: 2 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.085945] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.085946] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.085948] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.085954] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.086030] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.086032] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.086034] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.086036] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.086037] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.086039] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.086040] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.086041] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.086043] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.086044] CR2: 00007bb19a83a058 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.086046] PKRU: 55555554
<4> [515.086047] Call Trace:
<4> [515.086048] <TASK>
<4> [515.086052] ? lock_acquire+0xb0/0x300
<4> [515.086057] ? lock_release+0xd0/0x2b0
<4> [515.086062] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.086068] process_one_work+0x239/0x740
<4> [515.086075] worker_thread+0x200/0x3f0
<4> [515.086078] ? __pfx_worker_thread+0x10/0x10
<4> [515.086081] kthread+0x10d/0x150
<4> [515.086083] ? __pfx_kthread+0x10/0x10
<4> [515.086086] ret_from_fork+0x3bd/0x470
<4> [515.086089] ? __pfx_kthread+0x10/0x10
<4> [515.086092] ret_from_fork_asm+0x1a/0x30
<4> [515.086099] </TASK>
<4> [515.086101] irq event stamp: 1835385
<4> [515.086102] hardirqs last enabled at (1835391): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.086104] hardirqs last disabled at (1835396): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.086106] softirqs last enabled at (1834482): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.086109] softirqs last disabled at (1834467): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.086112] ---[ end trace 0000000000000000 ]---
<6> [515.086114] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.086193] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.086199] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.086446] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.086658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.086754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.086861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.086952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.087041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.087129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.087217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.087306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.087390] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.087491] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.087585] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.088629] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.098815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.099079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.100040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.100128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.100209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.100290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.100368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.100444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.100520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.100597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.100673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.100749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.100830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.100925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.101002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.101079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.101155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.101229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.101304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.101379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.101457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.101544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.101629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.101711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.101791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.101909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.101992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.102070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.102147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.102225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.102306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.102387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.102467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.102546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.102640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.102720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.102805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.102917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.103010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.103092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.103172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.103250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.103327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.103423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.103499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.103576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.103655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.103731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.104018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.104113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.104198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.104280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.104363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.104442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.104522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.104601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.104678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.104757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.104857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.104932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.105008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.105087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.105162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.105239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.105315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.105391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.105468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.105545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.105620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.105694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.105773] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.105909] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.105914] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61558, lrc_seqno=61558, guc_id=0, flags=0x73 in no process [-1]
<7> [515.105916] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.105985] ------------[ cut here ]------------
<4> [515.105986] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.105987] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:13/2860
<4> [515.106066] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.106133] [last unloaded: snd_hda_intel]
<4> [515.106136] CPU: 2 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.106139] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.106140] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.106141] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.106147] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.106223] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.106225] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.106227] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.106229] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.106230] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.106231] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.106232] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.106234] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.106235] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.106237] CR2: 00007bb19a83a058 CR3: 000000000364a003 CR4: 0000000000f72ef0
<4> [515.106238] PKRU: 55555554
<4> [515.106240] Call Trace:
<4> [515.106241] <TASK>
<4> [515.106245] ? lock_acquire+0xb0/0x300
<4> [515.106250] ? lock_release+0xd0/0x2b0
<4> [515.106255] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.106261] process_one_work+0x239/0x740
<4> [515.106267] worker_thread+0x200/0x3f0
<4> [515.106271] ? __pfx_worker_thread+0x10/0x10
<4> [515.106273] kthread+0x10d/0x150
<4> [515.106276] ? __pfx_kthread+0x10/0x10
<4> [515.106279] ret_from_fork+0x3bd/0x470
<4> [515.106281] ? __pfx_kthread+0x10/0x10
<4> [515.106284] ret_from_fork_asm+0x1a/0x30
<4> [515.106291] </TASK>
<4> [515.106293] irq event stamp: 1838609
<4> [515.106294] hardirqs last enabled at (1838615): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.106296] hardirqs last disabled at (1838620): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.106298] softirqs last enabled at (1837780): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.106301] softirqs last disabled at (1837769): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.106303] ---[ end trace 0000000000000000 ]---
<5> [515.107673] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61613, lrc_seqno=61613, guc_id=0, flags=0x73 in no process [-1]
<7> [515.107676] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.107747] ------------[ cut here ]------------
<4> [515.107748] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.107750] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:13/2860
<4> [515.107837] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.107921] [last unloaded: snd_hda_intel]
<4> [515.107924] CPU: 2 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.107927] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.107928] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.107929] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.107935] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.108011] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.108014] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.108017] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.108018] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.108019] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.108021] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.108022] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.108023] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.108025] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.108026] CR2: 00007bb19a83a058 CR3: 000000000364a003 CR4: 0000000000f72ef0
<4> [515.108027] PKRU: 55555554
<4> [515.108028] Call Trace:
<4> [515.108029] <TASK>
<4> [515.108033] ? lock_acquire+0xb0/0x300
<4> [515.108038] ? lock_release+0xd0/0x2b0
<4> [515.108043] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.108048] process_one_work+0x239/0x740
<4> [515.108055] worker_thread+0x200/0x3f0
<4> [515.108058] ? __pfx_worker_thread+0x10/0x10
<4> [515.108060] kthread+0x10d/0x150
<4> [515.108063] ? __pfx_kthread+0x10/0x10
<4> [515.108066] ret_from_fork+0x3bd/0x470
<4> [515.108068] ? __pfx_kthread+0x10/0x10
<4> [515.108071] ret_from_fork_asm+0x1a/0x30
<4> [515.108078] </TASK>
<4> [515.108079] irq event stamp: 1840529
<4> [515.108080] hardirqs last enabled at (1840535): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.108083] hardirqs last disabled at (1840540): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.108084] softirqs last enabled at (1837780): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.108087] softirqs last disabled at (1837769): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.108090] ---[ end trace 0000000000000000 ]---
<7> [515.108106] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.108217] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [515.108321] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.108399] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.108406] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.108457] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.108750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.108869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.108970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.109061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.109151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.109238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.109326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.109415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.109500] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.109604] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.109698] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.110746] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.120815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.121077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.122143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.122229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.122312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.122397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.122478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.122556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.122635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.122714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.122788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.122881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.122957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.123031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.123105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.123181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.123256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.123332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.123407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.123483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.123559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.123647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.123732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.123821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.123903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.123983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.124062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.124138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.124215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.124290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.124364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.124443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.124523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.124600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.124678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.124756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.124850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.124933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.125017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.125096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.125175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.125252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.125329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.125406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.125482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.125559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.125637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.125714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.125791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.125888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.125965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.126044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.126124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.126201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.126278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.126354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.126431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.126509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.126586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.126661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.126736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.126822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.126899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.126976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.127052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.127129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.127207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.127285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.127362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.127438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.127517] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.127640] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.127645] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61613, lrc_seqno=61613, guc_id=0, flags=0x73 in no process [-1]
<7> [515.127647] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.127715] ------------[ cut here ]------------
<4> [515.127716] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.127718] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:43/4174
<4> [515.127797] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.127872] [last unloaded: snd_hda_intel]
<4> [515.127876] CPU: 1 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.127879] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.127880] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.127881] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.127887] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.127964] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.127965] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.127968] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.127969] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.127971] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.127972] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.127973] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.127975] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.127976] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.127978] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.127979] PKRU: 55555554
<4> [515.127980] Call Trace:
<4> [515.127981] <TASK>
<4> [515.127986] ? lock_acquire+0xb0/0x300
<4> [515.127991] ? lock_release+0xd0/0x2b0
<4> [515.127996] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.128002] process_one_work+0x239/0x740
<4> [515.128009] worker_thread+0x200/0x3f0
<4> [515.128012] ? __pfx_worker_thread+0x10/0x10
<4> [515.128015] kthread+0x10d/0x150
<4> [515.128017] ? __pfx_kthread+0x10/0x10
<4> [515.128020] ret_from_fork+0x3bd/0x470
<4> [515.128023] ? __pfx_kthread+0x10/0x10
<4> [515.128026] ret_from_fork_asm+0x1a/0x30
<4> [515.128033] </TASK>
<4> [515.128034] irq event stamp: 414153
<4> [515.128036] hardirqs last enabled at (414159): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.128038] hardirqs last disabled at (414164): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.128040] softirqs last enabled at (413256): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.128043] softirqs last disabled at (413249): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.128045] ---[ end trace 0000000000000000 ]---
<6> [515.128047] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.128123] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.128129] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.129039] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.129156] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.129267] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.129491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.129594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.129698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.129796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.129907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.130001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.130089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.130181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.130264] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.130366] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.130461] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.131501] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.141816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.142085] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.143428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.143519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.143604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.143690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.143771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.143859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.143936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.144012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.144089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.144165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.144240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.144315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.144390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.144466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.144541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.144617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.144694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.144768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.144855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.144942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.145028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.145109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.145190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.145271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.145351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.145429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.145507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.145585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.145660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.145740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.145822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.145900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.145980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.146059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.146143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.146228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.146311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.146391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.146470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.146548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.146626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.146702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.146778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.146864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.146944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.147022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.147100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.147177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.147253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.147331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.147410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.147488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.147566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.147643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.147720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.147798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.147888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.147965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.148040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.148119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.148196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.148277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.148356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.148433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.148512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.148590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.148667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.148744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.148826] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.148961] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.148966] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61613, lrc_seqno=61613, guc_id=0, flags=0x73 in no process [-1]
<7> [515.148969] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.149038] ------------[ cut here ]------------
<4> [515.149039] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.149041] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:13/2860
<4> [515.149121] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.149193] [last unloaded: snd_hda_intel]
<4> [515.149197] CPU: 6 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.149201] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.149202] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.149204] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.149210] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.149289] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.149291] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.149293] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.149295] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.149297] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.149298] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.149300] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.149302] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [515.149304] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.149305] CR2: 00007ffdd492ffc0 CR3: 000000012f1a0001 CR4: 0000000000f72ef0
<4> [515.149307] PKRU: 55555554
<4> [515.149308] Call Trace:
<4> [515.149309] <TASK>
<4> [515.149314] ? lock_acquire+0xb0/0x300
<4> [515.149320] ? lock_release+0xd0/0x2b0
<4> [515.149324] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.149331] process_one_work+0x239/0x740
<4> [515.149339] worker_thread+0x200/0x3f0
<4> [515.149342] ? __pfx_worker_thread+0x10/0x10
<4> [515.149345] kthread+0x10d/0x150
<4> [515.149347] ? __pfx_kthread+0x10/0x10
<4> [515.149351] ret_from_fork+0x3bd/0x470
<4> [515.149354] ? __pfx_kthread+0x10/0x10
<4> [515.149357] ret_from_fork_asm+0x1a/0x30
<4> [515.149365] </TASK>
<4> [515.149366] irq event stamp: 1849213
<4> [515.149368] hardirqs last enabled at (1849219): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.149371] hardirqs last disabled at (1849224): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.149372] softirqs last enabled at (1848296): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.149376] softirqs last disabled at (1848289): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.149379] ---[ end trace 0000000000000000 ]---
<5> [515.150382] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61668, lrc_seqno=61668, guc_id=0, flags=0x73 in no process [-1]
<7> [515.150388] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.150494] ------------[ cut here ]------------
<4> [515.150495] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.150496] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:43/4174
<4> [515.150581] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core
<7> [515.150569] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [515.150624] intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [515.150651] CPU: 2 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.150654] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.150655] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.150656] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.150661] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<7> [515.150682] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [515.150739] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.150741] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.150743] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.150744] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.150745] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.150746] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.150747] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.150748] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.150749] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.150750] CR2: 00007bb19a83a058 CR3: 0000000132ea1006 CR4: 0000000000f72ef0
<4> [515.150752] PKRU: 55555554
<4> [515.150753] Call Trace:
<4> [515.150753] <TASK>
<4> [515.150757] ? lock_acquire+0xb0/0x300
<4> [515.150762] ? lock_release+0xd0/0x2b0
<4> [515.150767] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.150773] process_one_work+0x239/0x740
<4> [515.150780] worker_thread+0x200/0x3f0
<4> [515.150782] ? __pfx_worker_thread+0x10/0x10
<4> [515.150785] kthread+0x10d/0x150
<4> [515.150787] ? __pfx_kthread+0x10/0x10
<4> [515.150790] ret_from_fork+0x3bd/0x470
<4> [515.150793] ? __pfx_kthread+0x10/0x10
<4> [515.150795] ret_from_fork_asm+0x1a/0x30
<4> [515.150803] </TASK>
<4> [515.150804] irq event stamp: 421573
<4> [515.150811] hardirqs last enabled at (421579): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.150813] hardirqs last disabled at (421594): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.150815] softirqs last enabled at (421588): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.150818] softirqs last disabled at (421581): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.150821] ---[ end trace 0000000000000000 ]---
<6> [515.150823] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.150900] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.150907] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.151170] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.151390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.151489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.151586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.151678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.151766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.151869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.151961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.152050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.152134] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.152237] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.152333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.153373] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.163816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.164087] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.165456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.165542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.165624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.165706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.165785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.165888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.165966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.166042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.166118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.166195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.166270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.166346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.166421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.166497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.166573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.166648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.166723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.166798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.166888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.166976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.167062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.167148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.167231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.167312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.167392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.167470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.167548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.167624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.167700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.167778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.167867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.167946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.168025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.168106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.168192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.168281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.168368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.168453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.168535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.168613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.168692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.168769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.168855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.168934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.169013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.169090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.169168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.169245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.169322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.169399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.169479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.169556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.169634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.169710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.169785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.169872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.169949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.170025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.170101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.170180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.170257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.170336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.170414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.170490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.170568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.170644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.170720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.170796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.170887] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.171009] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.171013] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61668, lrc_seqno=61668, guc_id=0, flags=0x73 in no process [-1]
<7> [515.171015] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.171084] ------------[ cut here ]------------
<4> [515.171086] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.171087] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:13/2860
<4> [515.171167] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.171231] [last unloaded: snd_hda_intel]
<4> [515.171235] CPU: 6 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.171237] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.171239] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.171240] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.171245] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.171323] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.171325] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.171327] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.171328] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.171330] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.171331] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.171332] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.171334] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [515.171335] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.171337] CR2: 00007ffdd492ffc0 CR3: 000000012f1a0001 CR4: 0000000000f72ef0
<4> [515.171338] PKRU: 55555554
<4> [515.171339] Call Trace:
<4> [515.171341] <TASK>
<4> [515.171344] ? lock_acquire+0xb0/0x300
<4> [515.171349] ? lock_release+0xd0/0x2b0
<4> [515.171354] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.171360] process_one_work+0x239/0x740
<4> [515.171367] worker_thread+0x200/0x3f0
<4> [515.171370] ? __pfx_worker_thread+0x10/0x10
<4> [515.171372] kthread+0x10d/0x150
<4> [515.171375] ? __pfx_kthread+0x10/0x10
<4> [515.171378] ret_from_fork+0x3bd/0x470
<4> [515.171380] ? __pfx_kthread+0x10/0x10
<4> [515.171383] ret_from_fork_asm+0x1a/0x30
<4> [515.171390] </TASK>
<4> [515.171392] irq event stamp: 1853649
<4> [515.171393] hardirqs last enabled at (1853655): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.171395] hardirqs last disabled at (1853660): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.171397] softirqs last enabled at (1852640): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.171399] softirqs last disabled at (1852633): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.171402] ---[ end trace 0000000000000000 ]---
<6> [515.171404] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.171479] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.171484] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.172390] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.172505] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.172617] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.172883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.172986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.173087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.173179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.173267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.173353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.173440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.173528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.173611] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.173713] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.173818] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.174864] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.184814] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.185079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.186149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.186239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.186323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.186406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.186489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.186569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.186649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.186729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.186813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.186891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.186967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.187043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.187119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.187195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.187271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.187347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.187422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.187497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.187576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.187664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.187749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.187837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.187917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.187999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.188079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.188157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.188235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.188313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.188389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.188473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.188556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.188636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.188718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.188798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.188897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.188985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.189070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.189152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.189233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.189312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.189390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.189468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.189545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.189623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.189702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.189780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.189867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.189947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.190025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.190105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.190187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.190264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.190343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.190419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.190496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.190575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.190655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.190732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.190810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.190888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.190964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.191041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.191118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.191194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.191273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.191350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.191425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.191500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.191579] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.191704] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.191709] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61668, lrc_seqno=61668, guc_id=0, flags=0x73 in no process [-1]
<7> [515.191711] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.191780] ------------[ cut here ]------------
<4> [515.191781] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.191783] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:45/4176
<4> [515.191870] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.191940] [last unloaded: snd_hda_intel]
<4> [515.191944] CPU: 2 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.191947] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.191948] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.191950] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.191956] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.192032] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.192034] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.192036] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.192038] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.192039] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.192040] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.192042] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.192043] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.192045] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.192046] CR2: 00007bb19a83a058 CR3: 0000000132ea1006 CR4: 0000000000f72ef0
<4> [515.192047] PKRU: 55555554
<4> [515.192049] Call Trace:
<4> [515.192050] <TASK>
<4> [515.192054] ? lock_acquire+0xb0/0x300
<4> [515.192060] ? lock_release+0xd0/0x2b0
<4> [515.192065] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.192071] process_one_work+0x239/0x740
<4> [515.192079] worker_thread+0x200/0x3f0
<4> [515.192082] ? __pfx_worker_thread+0x10/0x10
<4> [515.192085] kthread+0x10d/0x150
<4> [515.192087] ? __pfx_kthread+0x10/0x10
<4> [515.192090] ret_from_fork+0x3bd/0x470
<4> [515.192093] ? __pfx_kthread+0x10/0x10
<4> [515.192095] ret_from_fork_asm+0x1a/0x30
<4> [515.192103] </TASK>
<4> [515.192105] irq event stamp: 404369
<4> [515.192106] hardirqs last enabled at (404375): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.192108] hardirqs last disabled at (404380): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.192110] softirqs last enabled at (403388): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.192113] softirqs last disabled at (403381): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.192115] ---[ end trace 0000000000000000 ]---
<7> [515.193021] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.193140] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [515.193191] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61723, lrc_seqno=61723, guc_id=0, flags=0x73 in no process [-1]
<7> [515.193195] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.193272] ------------[ cut here ]------------
<4> [515.193274] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.193275] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:43/4174
<4> [515.193361] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.193426] [last unloaded: snd_hda_intel]
<4> [515.193430] CPU: 1 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.193433] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.193434] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.193435] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.193441] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.193522] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.193524] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.193527] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.193528] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.193529] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.193530] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.193532] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.193533] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.193535] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.193536] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.193538] PKRU: 55555554
<4> [515.193539] Call Trace:
<4> [515.193540] <TASK>
<4> [515.193544] ? lock_acquire+0xb0/0x300
<4> [515.193550] ? lock_release+0xd0/0x2b0
<4> [515.193555] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.193561] process_one_work+0x239/0x740
<4> [515.193568] worker_thread+0x200/0x3f0
<4> [515.193571] ? __pfx_worker_thread+0x10/0x10
<4> [515.193573] kthread+0x10d/0x150
<4> [515.193576] ? __pfx_kthread+0x10/0x10
<4> [515.193579] ret_from_fork+0x3bd/0x470
<4> [515.193581] ? __pfx_kthread+0x10/0x10
<4> [515.193584] ret_from_fork_asm+0x1a/0x30
<4> [515.193591] </TASK>
<4> [515.193592] irq event stamp: 428691
<4> [515.193594] hardirqs last enabled at (428697): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.193596] hardirqs last disabled at (428702): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.193598] softirqs last enabled at (427034): [<ffffffff8133bdb3>] kernel_fpu_end+0x53/0x70
<4> [515.193600] softirqs last disabled at (427032): [<ffffffff8133c4c4>] kernel_fpu_begin_mask+0xc4/0x120
<4> [515.193603] ---[ end trace 0000000000000000 ]---
<6> [515.193605] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.193686] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.193693] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.193871] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.194096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.194199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.194302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.194396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.194487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.194574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.194660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.194750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.194840] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.194942] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.195038] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.196076] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.205815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.206076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.207096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.207184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.207267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.207346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.207426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.207502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.207577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.207654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.207731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.207811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.207887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.207964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.208038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.208114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.208188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.208264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.208339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.208413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.208488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.208575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.208660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.208742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.208826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.208907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.208986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.209063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.209140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.209218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.209296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.209376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.209456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.209535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.209615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.209694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.209779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.209873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.209958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.210039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.210120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.210199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.210281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.210360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.210438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.210516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.210594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.210672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.210749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.210830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.210906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.210985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.211065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.211142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.211219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.211295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.211371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.211448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.211525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.211600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.211675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.211753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.211836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.211915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.211992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.212069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.212147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.212226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.212302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.212378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.212458] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.212582] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.212587] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61723, lrc_seqno=61723, guc_id=0, flags=0x73 in no process [-1]
<7> [515.212589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.212658] ------------[ cut here ]------------
<4> [515.212659] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.212660] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:13/2860
<4> [515.212751] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.212823] [last unloaded: snd_hda_intel]
<4> [515.212827] CPU: 8 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.212831] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.212832] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.212833] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.212839] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.212917] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.212919] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.212921] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.212923] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.212924] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.212925] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.212926] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.212928] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.212930] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.212931] CR2: 0000727e7460d098 CR3: 000000012f1a0003 CR4: 0000000000f72ef0
<4> [515.212932] PKRU: 55555554
<4> [515.212934] Call Trace:
<4> [515.212935] <TASK>
<4> [515.212939] ? lock_acquire+0xb0/0x300
<4> [515.212944] ? lock_release+0xd0/0x2b0
<4> [515.212949] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.212955] process_one_work+0x239/0x740
<4> [515.212962] worker_thread+0x200/0x3f0
<4> [515.212965] ? __pfx_worker_thread+0x10/0x10
<4> [515.212968] kthread+0x10d/0x150
<4> [515.212970] ? __pfx_kthread+0x10/0x10
<4> [515.212973] ret_from_fork+0x3bd/0x470
<4> [515.212976] ? __pfx_kthread+0x10/0x10
<4> [515.212978] ret_from_fork_asm+0x1a/0x30
<4> [515.212986] </TASK>
<4> [515.212987] irq event stamp: 1861589
<4> [515.212989] hardirqs last enabled at (1861595): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.212991] hardirqs last disabled at (1861600): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.212993] softirqs last enabled at (1860428): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.212996] softirqs last disabled at (1860423): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.212998] ---[ end trace 0000000000000000 ]---
<6> [515.213814] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.213897] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.213906] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.213921] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.214030] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.214140] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.214361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.214463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.214564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.214656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.214744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.214849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.214937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.215026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.215110] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.215211] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.215307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.216349] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.226816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.227090] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.228167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.228259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.228343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.228426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.228507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.228587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.228666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.228744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.228832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.228912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.228990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.229066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.229144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.229220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.229296] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.229371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.229446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.229520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.229594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.229683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.229769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.229866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.229949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.230031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.230114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.230194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.230273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.230349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.230426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.230505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.230585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.230663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.230742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.230827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.230912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.230997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.231081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.231162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.231242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.231319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.231396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.231472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.231547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.231625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.231703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.231780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.231878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.231956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.232031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.232108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.232187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.232265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.232341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.232417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.232495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.232572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.232649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.232725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.232800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.232895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.232973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.233051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.233128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.233204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.233280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.233355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.233428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.233503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.233580] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.233704] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.233709] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61723, lrc_seqno=61723, guc_id=0, flags=0x73 in no process [-1]
<7> [515.233712] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.233781] ------------[ cut here ]------------
<4> [515.233782] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.233784] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:45/4176
<4> [515.233880] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.234028] [last unloaded: snd_hda_intel]
<4> [515.234035] CPU: 1 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.234040] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.234043] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.234046] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.234058] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.234171] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.234174] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.234178] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.234180] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.234182] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.234184] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.234187] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.234189] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.234192] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.234194] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.234196] PKRU: 55555554
<4> [515.234198] Call Trace:
<4> [515.234201] <TASK>
<4> [515.234208] ? lock_acquire+0xb0/0x300
<4> [515.234217] ? lock_release+0xd0/0x2b0
<4> [515.234226] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.234238] process_one_work+0x239/0x740
<4> [515.234249] worker_thread+0x200/0x3f0
<4> [515.234254] ? __pfx_worker_thread+0x10/0x10
<4> [515.234259] kthread+0x10d/0x150
<4> [515.234263] ? __pfx_kthread+0x10/0x10
<4> [515.234268] ret_from_fork+0x3bd/0x470
<4> [515.234272] ? __pfx_kthread+0x10/0x10
<4> [515.234276] ret_from_fork_asm+0x1a/0x30
<4> [515.234290] </TASK>
<4> [515.234292] irq event stamp: 408901
<4> [515.234294] hardirqs last enabled at (408907): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.234298] hardirqs last disabled at (408912): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.234301] softirqs last enabled at (408002): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.234305] softirqs last disabled at (407995): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.234309] ---[ end trace 0000000000000000 ]---
<5> [515.234959] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61778, lrc_seqno=61778, guc_id=0, flags=0x73 in no process [-1]
<7> [515.234974] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.235051] ------------[ cut here ]------------
<4> [515.235052] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.235054] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [515.235261] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.235335] [last unloaded: snd_hda_intel]
<4> [515.235340] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.235345] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.235346] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.235347] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.235355] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.235444] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.235447] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.235450] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.235451] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.235453] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.235454] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.235455] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.235457] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.235459] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.235462] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.235463] PKRU: 55555554
<4> [515.235464] Call Trace:
<4> [515.235466] <TASK>
<4> [515.235470] ? lock_acquire+0xb0/0x300
<4> [515.235477] ? lock_release+0xd0/0x2b0
<4> [515.235483] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.235490] process_one_work+0x239/0x740
<4> [515.235497] worker_thread+0x200/0x3f0
<4> [515.235501] ? __pfx_worker_thread+0x10/0x10
<4> [515.235504] kthread+0x10d/0x150
<4> [515.235507] ? __pfx_kthread+0x10/0x10
<4> [515.235510] ret_from_fork+0x3bd/0x470
<4> [515.235513] ? __pfx_kthread+0x10/0x10
<4> [515.235515] ret_from_fork_asm+0x1a/0x30
<4> [515.235523] </TASK>
<4> [515.235524] irq event stamp: 410627
<4> [515.235525] hardirqs last enabled at (410633): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.235528] hardirqs last disabled at (410638): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.235530] softirqs last enabled at (408002): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.235533] softirqs last disabled at (407995): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.235536] ---[ end trace 0000000000000000 ]---
<6> [515.235538] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.235637] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.235658] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.235879] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.235999] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.236114] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.236326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.236425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.236527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.236621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.236711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.236798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.236899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.236987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.237069] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.237171] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.237266] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.238306] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.248815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.249076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.250090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.250184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.250267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.250349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.250429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.250507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.250583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.250659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.250733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.250812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.250888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.250965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.251041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.251117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.251193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.251268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.251343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.251420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.251495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.251582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.251667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.251749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.251857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.251938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.252018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.252096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.252173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.252250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.252326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.252406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.252486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.252566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.252645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.252725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.252813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.252898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.252982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.253064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.253144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.253221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.253298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.253373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.253449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.253527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.253606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.253683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.253761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.253846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.253929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.254010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.254092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.254172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.254251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.254329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.254406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.254484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.254561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.254637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.254712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.254791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.254878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.254957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.255034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.255110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.255187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.255263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.255338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.255413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.255492] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.255614] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.255619] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61778, lrc_seqno=61778, guc_id=0, flags=0x73 in no process [-1]
<7> [515.255621] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.255689] ------------[ cut here ]------------
<4> [515.255691] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.255692] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<4> [515.255771] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.256129] [last unloaded: snd_hda_intel]
<4> [515.256132] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.256135] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.256137] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.256138] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.256144] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.256223] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.256225] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.256227] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.256229] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.256230] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.256231] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.256232] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.256234] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.256236] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.256237] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.256238] PKRU: 55555554
<4> [515.256240] Call Trace:
<4> [515.256241] <TASK>
<4> [515.256245] ? lock_acquire+0xb0/0x300
<4> [515.256250] ? lock_release+0xd0/0x2b0
<4> [515.256255] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.256261] process_one_work+0x239/0x740
<4> [515.256268] worker_thread+0x200/0x3f0
<4> [515.256271] ? __pfx_worker_thread+0x10/0x10
<4> [515.256274] kthread+0x10d/0x150
<4> [515.256277] ? __pfx_kthread+0x10/0x10
<4> [515.256280] ret_from_fork+0x3bd/0x470
<4> [515.256282] ? __pfx_kthread+0x10/0x10
<4> [515.256285] ret_from_fork_asm+0x1a/0x30
<4> [515.256293] </TASK>
<4> [515.256294] irq event stamp: 413909
<4> [515.256295] hardirqs last enabled at (413915): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.256297] hardirqs last disabled at (413920): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.256299] softirqs last enabled at (412934): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.256302] softirqs last disabled at (412927): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.256305] ---[ end trace 0000000000000000 ]---
<6> [515.256307] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.256383] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.256389] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.257312] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.257429] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.257539] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.257764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.257878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.257980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.258075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.258166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.258254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.258344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.258435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.258521] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.258625] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.258720] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.259767] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.270072] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.270339] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.271419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.271515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.271601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.271686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.271767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.271866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.271948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.272028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.272107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.272185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.272262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.272339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.272416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.272493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.272569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.272646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.272722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.272798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.272889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.272976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.273061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.273144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.273225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.273307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.273388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.273468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.273550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.273631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.273710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.273794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.273894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.273975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.274054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.274133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.274218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.274303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.274387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.274468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.274548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.274625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.274702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.274779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.274869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.274948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.275030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.275108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.275185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.275261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.275337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.275415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.275494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.275572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.275649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.275725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.275800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.275896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.275974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.276050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.276125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.276204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.276280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.276357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.276433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.276508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.276584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.276661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.276735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.276813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.276892] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.277016] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.277020] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61778, lrc_seqno=61778, guc_id=0, flags=0x73 in no process [-1]
<7> [515.277023] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.277092] ------------[ cut here ]------------
<4> [515.277093] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.277094] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [515.277173] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.277241] [last unloaded: snd_hda_intel]
<4> [515.277245] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.277248] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.277249] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.277250] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.277256] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.277332] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.277334] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.277336] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.277338] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.277339] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.277340] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.277342] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.277343] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.277345] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.277346] CR2: 0000727e7e545c60 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [515.277348] PKRU: 55555554
<4> [515.277349] Call Trace:
<4> [515.277350] <TASK>
<4> [515.277354] ? lock_acquire+0xb0/0x300
<4> [515.277360] ? lock_release+0xd0/0x2b0
<4> [515.277365] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.277371] process_one_work+0x239/0x740
<4> [515.277378] worker_thread+0x200/0x3f0
<4> [515.277382] ? __pfx_worker_thread+0x10/0x10
<4> [515.277384] kthread+0x10d/0x150
<4> [515.277387] ? __pfx_kthread+0x10/0x10
<4> [515.277390] ret_from_fork+0x3bd/0x470
<4> [515.277392] ? __pfx_kthread+0x10/0x10
<4> [515.277395] ret_from_fork_asm+0x1a/0x30
<4> [515.277403] </TASK>
<4> [515.277404] irq event stamp: 1870383
<4> [515.277405] hardirqs last enabled at (1870389): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.277408] hardirqs last disabled at (1870394): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.277410] softirqs last enabled at (1869540): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.277413] softirqs last disabled at (1869533): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.277415] ---[ end trace 0000000000000000 ]---
<5> [515.278599] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61833, lrc_seqno=61833, guc_id=0, flags=0x73 in no process [-1]
<7> [515.278603] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.278683] ------------[ cut here ]------------
<4> [515.278684] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.278686] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:45/4176
<7> [515.278671] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [515.278770] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake
<7> [515.278787] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [515.278845] idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: snd_hda_intel]
<4> [515.278857] CPU: 8 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.278860] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.278861] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.278862] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.278868] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.278948] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.278950] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.278952] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.278953] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.278955] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.278956] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.278957] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.278958] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [515.278960] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.278961] CR2: 0000727e7460d098 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.278963] PKRU: 55555554
<4> [515.278964] Call Trace:
<4> [515.278965] <TASK>
<4> [515.278969] ? lock_acquire+0xb0/0x300
<4> [515.278975] ? lock_release+0xd0/0x2b0
<4> [515.278980] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.278986] process_one_work+0x239/0x740
<4> [515.278993] worker_thread+0x200/0x3f0
<4> [515.278996] ? __pfx_worker_thread+0x10/0x10
<4> [515.278999] kthread+0x10d/0x150
<4> [515.279001] ? __pfx_kthread+0x10/0x10
<4> [515.279004] ret_from_fork+0x3bd/0x470
<4> [515.279006] ? __pfx_kthread+0x10/0x10
<4> [515.279009] ret_from_fork_asm+0x1a/0x30
<4> [515.279017] </TASK>
<4> [515.279018] irq event stamp: 421043
<4> [515.279019] hardirqs last enabled at (421049): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.279021] hardirqs last disabled at (421054): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.279023] softirqs last enabled at (420514): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.279026] softirqs last disabled at (420509): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.279028] ---[ end trace 0000000000000000 ]---
<6> [515.279030] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.279109] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.279115] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.279295] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.279511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.279607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.279705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.279797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.279909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.279998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.280088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.280180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.280267] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.280369] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.280463] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.281506] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.291813] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.292076] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.293119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.293201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.293275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.293348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.293420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.293489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.293559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.293628] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.293698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.293768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.293850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.293925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.293993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.294061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.294129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.294197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.294264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.294331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.294398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.294475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.294552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.294626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.294699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.294773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.294869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.294948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.295023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.295095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.295167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.295241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.295315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.295387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.295460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.295532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.295610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.295694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.295780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.295877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.295962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.296042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.296121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.296201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.296306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.296399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.296498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.296592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.296671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.296750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.296835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.296916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.296990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.297061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.297131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.297202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.297272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.297343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.297414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.297484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.297553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.297626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.297697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.297769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.297852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.297929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.297999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.298071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.298141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.298210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.298285] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.298404] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.298408] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61833, lrc_seqno=61833, guc_id=0, flags=0x73 in no process [-1]
<7> [515.298410] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.298473] ------------[ cut here ]------------
<4> [515.298474] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.298475] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [515.298548] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.298610] [last unloaded: snd_hda_intel]
<4> [515.298613] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.298615] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.298617] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.298618] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.298623] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.298698] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.298700] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.298702] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.298703] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.298704] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.298705] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.298706] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.298708] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.298709] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.298710] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.298712] PKRU: 55555554
<4> [515.298713] Call Trace:
<4> [515.298714] <TASK>
<4> [515.298718] ? lock_acquire+0xb0/0x300
<4> [515.298723] ? lock_release+0xd0/0x2b0
<4> [515.298728] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.298733] process_one_work+0x239/0x740
<4> [515.298740] worker_thread+0x200/0x3f0
<4> [515.298743] ? __pfx_worker_thread+0x10/0x10
<4> [515.298745] kthread+0x10d/0x150
<4> [515.298747] ? __pfx_kthread+0x10/0x10
<4> [515.298750] ret_from_fork+0x3bd/0x470
<4> [515.298753] ? __pfx_kthread+0x10/0x10
<4> [515.298755] ret_from_fork_asm+0x1a/0x30
<4> [515.298762] </TASK>
<4> [515.298763] irq event stamp: 1874269
<4> [515.298764] hardirqs last enabled at (1874275): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.298766] hardirqs last disabled at (1874280): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.298768] softirqs last enabled at (1873222): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.298771] softirqs last disabled at (1873215): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.298773] ---[ end trace 0000000000000000 ]---
<6> [515.299612] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.299690] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.299697] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.299711] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.300262] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.300369] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.300584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.300679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.300771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.300880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.300969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.301048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.301127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.301207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.301283] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.301374] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.301462] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.302486] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.312818] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.313086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.314238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.314346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.314431] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.314512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.314591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.314668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.314744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.314826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.314909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.314990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.315070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.315150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.315230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.315310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.315389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.315469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.315548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.315629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.315709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.315800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.315906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.315995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.316081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.316167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.316253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.316336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.316420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.316503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.316586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.316672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.316757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.316849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.316935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.317019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.317108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.317198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.317293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.317383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.317470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.317554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.317638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.317719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.317800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.317896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.317982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.318065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.318148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.318230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.318312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.318395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.318480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.318562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.318644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.318725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.318809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.318892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.318974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.319054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.319133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.319216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.319297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.319379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.319460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.319541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.319624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.319706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.319786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.319877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.319961] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.320089] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.320094] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61833, lrc_seqno=61833, guc_id=0, flags=0x73 in no process [-1]
<7> [515.320097] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.320169] ------------[ cut here ]------------
<4> [515.320171] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.320172] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:43/4174
<4> [515.320256] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.320325] [last unloaded: snd_hda_intel]
<4> [515.320329] CPU: 4 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.320332] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.320334] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.320335] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.320342] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.320423] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.320425] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.320428] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.320429] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.320430] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.320432] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.320433] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.320434] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [515.320436] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.320437] CR2: 00007df35c068018 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [515.320439] PKRU: 55555554
<4> [515.320440] Call Trace:
<4> [515.320441] <TASK>
<4> [515.320445] ? lock_acquire+0xb0/0x300
<4> [515.320452] ? lock_release+0xd0/0x2b0
<4> [515.320456] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.320463] process_one_work+0x239/0x740
<4> [515.320470] worker_thread+0x200/0x3f0
<4> [515.320473] ? __pfx_worker_thread+0x10/0x10
<4> [515.320476] kthread+0x10d/0x150
<4> [515.320479] ? __pfx_kthread+0x10/0x10
<4> [515.320482] ret_from_fork+0x3bd/0x470
<4> [515.320485] ? __pfx_kthread+0x10/0x10
<4> [515.320487] ret_from_fork_asm+0x1a/0x30
<4> [515.320495] </TASK>
<4> [515.320496] irq event stamp: 443225
<4> [515.320498] hardirqs last enabled at (443231): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.320500] hardirqs last disabled at (443236): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.320502] softirqs last enabled at (442222): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.320505] softirqs last disabled at (442215): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.320508] ---[ end trace 0000000000000000 ]---
<5> [515.321302] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61888, lrc_seqno=61888, guc_id=0, flags=0x73 in no process [-1]
<7> [515.321309] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.321428] ------------[ cut here ]------------
<4> [515.321430] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.321432] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:43/4174
<4> [515.321521] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.321590] [last unloaded: snd_hda_intel]
<4> [515.321593] CPU: 6 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.321596] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.321598] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.321599] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.321605] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.321686] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.321688] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.321690] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.321692] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.321693] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.321694] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.321695] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.321697] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [515.321698] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.321700] CR2: 00007ffdd492ffc0 CR3: 000000028bb80005 CR4: 0000000000f72ef0
<4> [515.321701] PKRU: 55555554
<4> [515.321702] Call Trace:
<4> [515.321703] <TASK>
<4> [515.321707] ? lock_acquire+0xb0/0x300
<4> [515.321713] ? lock_release+0xd0/0x2b0
<4> [515.321718] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<7> [515.321668] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [515.321725] process_one_work+0x239/0x740
<4> [515.321731] worker_thread+0x200/0x3f0
<4> [515.321734] ? __pfx_worker_thread+0x10/0x10
<4> [515.321737] kthread+0x10d/0x150
<4> [515.321739] ? __pfx_kthread+0x10/0x10
<4> [515.321742] ret_from_fork+0x3bd/0x470
<4> [515.321743] ? __pfx_kthread+0x10/0x10
<4> [515.321746] ret_from_fork_asm+0x1a/0x30
<4> [515.321753] </TASK>
<4> [515.321754] irq event stamp: 444409
<4> [515.321755] hardirqs last enabled at (444415): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.321757] hardirqs last disabled at (444420): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.321759] softirqs last enabled at (442222): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.321762] softirqs last disabled at (442215): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.321764] ---[ end trace 0000000000000000 ]---
<6> [515.321766] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [515.321785] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [515.321858] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.321865] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.321955] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.322160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.322249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.322339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.322424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.322506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.322587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.322669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.322751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.322832] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.322933] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.323019] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.324042] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.333813] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.334082] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.335141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.335233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.335317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.335400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.335483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.335566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.335645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.335723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.335800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.335905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.335982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.336059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.336136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.336213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.336289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.336364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.336439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.336514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.336590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.336676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.336761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.336853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.336933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.337014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.337096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.337174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.337253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.337334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.337411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.337492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.337572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.337650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.337729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.338222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.338314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.338399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.338483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.338563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.338643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.338721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.338798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.338889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.338965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.339043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.339121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.339203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.339286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.339365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.339441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.339520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.339601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.339678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.339756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.339839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.339915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.339993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.340070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.340145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.340222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.340302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.340379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.340457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.340534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.340611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.340689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.340767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.340853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.340928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.341007] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.341130] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.341134] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61888, lrc_seqno=61888, guc_id=0, flags=0x73 in no process [-1]
<7> [515.341137] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.341207] ------------[ cut here ]------------
<4> [515.341208] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.341209] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:43/4174
<4> [515.341290] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.341360] [last unloaded: snd_hda_intel]
<4> [515.341364] CPU: 6 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.341367] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.341368] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.341370] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.341376] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.341453] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.341455] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.341457] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.341459] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.341460] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.341462] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.341463] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.341465] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [515.341466] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.341468] CR2: 00007ffdd492ffc0 CR3: 000000028bb80005 CR4: 0000000000f72ef0
<4> [515.341470] PKRU: 55555554
<4> [515.341471] Call Trace:
<4> [515.341473] <TASK>
<4> [515.341477] ? lock_acquire+0xb0/0x300
<4> [515.341483] ? lock_release+0xd0/0x2b0
<4> [515.341488] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.341494] process_one_work+0x239/0x740
<4> [515.341501] worker_thread+0x200/0x3f0
<4> [515.341504] ? __pfx_worker_thread+0x10/0x10
<4> [515.341507] kthread+0x10d/0x150
<4> [515.341510] ? __pfx_kthread+0x10/0x10
<4> [515.341513] ret_from_fork+0x3bd/0x470
<4> [515.341516] ? __pfx_kthread+0x10/0x10
<4> [515.341519] ret_from_fork_asm+0x1a/0x30
<4> [515.341527] </TASK>
<4> [515.341528] irq event stamp: 447655
<4> [515.341529] hardirqs last enabled at (447661): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.341532] hardirqs last disabled at (447666): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.341534] softirqs last enabled at (446806): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.341537] softirqs last disabled at (446793): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.341540] ---[ end trace 0000000000000000 ]---
<6> [515.341542] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.341617] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.341626] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.341675] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.341906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.342002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.342102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.342234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.342344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.342440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.342529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.342619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.342704] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.342811] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.342908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.343948] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.354679] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.354938] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.355983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.356064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.356138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.356212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.356286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.356358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.356429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.356501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.356571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.356642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.356712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.356782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.356874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.356952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.357029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.357105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.357182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.357259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.357335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.357423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.357511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.357595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.357676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.357757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.357849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.357974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.358090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.358188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.358278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.358368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.358457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.358543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.358630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.358716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.358813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.358904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.358994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.359081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.359167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.359250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.359334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.359416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.359496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.359579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.359664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.359751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.359845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.359930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.360013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.360098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.360183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.360268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.360353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.360436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.360516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.360598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.360680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.360760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.360849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.360938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.361021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.361108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.361190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.361272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.361355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.361438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.361518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.361599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.361683] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.362865] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.362873] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61888, lrc_seqno=61888, guc_id=0, flags=0x73 in no process [-1]
<7> [515.362876] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.362956] ------------[ cut here ]------------
<4> [515.362958] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.362959] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:43/4174
<4> [515.363045] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.363114] [last unloaded: snd_hda_intel]
<4> [515.363117] CPU: 2 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.363120] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.363121] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.363123] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.363129] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.363208] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.363210] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.363213] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.363214] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.363215] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.363217] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.363218] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.363219] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.363221] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.363222] CR2: 00007bb19a83a058 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.363224] PKRU: 55555554
<4> [515.363225] Call Trace:
<4> [515.363226] <TASK>
<4> [515.363231] ? lock_acquire+0xb0/0x300
<4> [515.363237] ? lock_release+0xd0/0x2b0
<4> [515.363242] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.363247] process_one_work+0x239/0x740
<4> [515.363254] worker_thread+0x200/0x3f0
<4> [515.363257] ? __pfx_worker_thread+0x10/0x10
<4> [515.363260] kthread+0x10d/0x150
<4> [515.363263] ? __pfx_kthread+0x10/0x10
<4> [515.363266] ret_from_fork+0x3bd/0x470
<4> [515.363268] ? __pfx_kthread+0x10/0x10
<4> [515.363271] ret_from_fork_asm+0x1a/0x30
<4> [515.363279] </TASK>
<4> [515.363280] irq event stamp: 450827
<4> [515.363282] hardirqs last enabled at (450833): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.363284] hardirqs last disabled at (450838): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.363286] softirqs last enabled at (449756): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.363289] softirqs last disabled at (449751): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.363291] ---[ end trace 0000000000000000 ]---
<7> [515.363382] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.363524] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [515.363746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61943, lrc_seqno=61943, guc_id=0, flags=0x73 in no process [-1]
<7> [515.363749] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.363872] ------------[ cut here ]------------
<4> [515.363873] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.363874] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:43/4174
<4> [515.363952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.364015] [last unloaded: snd_hda_intel]
<4> [515.364018] CPU: 2 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.364021] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.364022] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.364023] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.364028] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.364103] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.364105] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.364107] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.364108] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.364110] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.364111] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.364112] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.364113] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.364115] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.364116] CR2: 00007bb19a83a058 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.364118] PKRU: 55555554
<4> [515.364119] Call Trace:
<4> [515.364120] <TASK>
<4> [515.364123] ? lock_acquire+0xb0/0x300
<4> [515.364129] ? lock_release+0xd0/0x2b0
<4> [515.364134] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.364139] process_one_work+0x239/0x740
<4> [515.364146] worker_thread+0x200/0x3f0
<4> [515.364149] ? __pfx_worker_thread+0x10/0x10
<4> [515.364152] kthread+0x10d/0x150
<4> [515.364154] ? __pfx_kthread+0x10/0x10
<4> [515.364157] ret_from_fork+0x3bd/0x470
<4> [515.364159] ? __pfx_kthread+0x10/0x10
<4> [515.364162] ret_from_fork_asm+0x1a/0x30
<4> [515.364170] </TASK>
<4> [515.364171] irq event stamp: 453419
<4> [515.364172] hardirqs last enabled at (453425): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.364174] hardirqs last disabled at (453430): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.364176] softirqs last enabled at (452664): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.364179] softirqs last disabled at (452377): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.364182] ---[ end trace 0000000000000000 ]---
<6> [515.364183] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.364258] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.364269] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.364317] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.364747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.364854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.364952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.365044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.365135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.365226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.365314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.365402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.365484] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.365585] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.365679] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.366716] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.376813] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.377075] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.378287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.378374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.378455] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.378537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.378617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.378696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.378775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.378901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.378984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.379063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.379139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.379215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.379291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.379367] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.379443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.379516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.379589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.379666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.379739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.379832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.379918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.380000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.380079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.380159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.380239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.380318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.380396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.380474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.380555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.380640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.380722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.380801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.380894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.380974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.381057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.381143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.381231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.381315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.381396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.381474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.381551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.381628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.381702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.381778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.381863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.381941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.382016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.382090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.382164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.382241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.382320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.382396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.382472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.382549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.382624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.382699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.382775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.382863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.382940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.383018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.383095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.383178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.383258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.383335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.383412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.383489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.383567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.383645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.383725] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.384106] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.384111] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61943, lrc_seqno=61943, guc_id=0, flags=0x73 in no process [-1]
<7> [515.384114] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.384187] ------------[ cut here ]------------
<4> [515.384188] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.384189] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:43/4174
<4> [515.384271] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.384337] [last unloaded: snd_hda_intel]
<4> [515.384340] CPU: 2 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.384343] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.384344] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.384345] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.384351] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.384430] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.384431] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.384434] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.384435] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.384436] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.384438] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.384439] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.384440] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.384442] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.384443] CR2: 00007bb19a83a058 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.384445] PKRU: 55555554
<4> [515.384446] Call Trace:
<4> [515.384447] <TASK>
<4> [515.384451] ? lock_acquire+0xb0/0x300
<4> [515.384457] ? lock_release+0xd0/0x2b0
<4> [515.384462] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.384468] process_one_work+0x239/0x740
<4> [515.384475] worker_thread+0x200/0x3f0
<4> [515.384478] ? __pfx_worker_thread+0x10/0x10
<4> [515.384481] kthread+0x10d/0x150
<4> [515.384483] ? __pfx_kthread+0x10/0x10
<4> [515.384486] ret_from_fork+0x3bd/0x470
<4> [515.384489] ? __pfx_kthread+0x10/0x10
<4> [515.384491] ret_from_fork_asm+0x1a/0x30
<4> [515.384499] </TASK>
<4> [515.384500] irq event stamp: 456777
<4> [515.384501] hardirqs last enabled at (456783): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.384504] hardirqs last disabled at (456788): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.384506] softirqs last enabled at (455598): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.384509] softirqs last disabled at (455591): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.384511] ---[ end trace 0000000000000000 ]---
<6> [515.384513] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.384590] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.384597] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.385698] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.385815] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.385928] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.386155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.386259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.386361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.386457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.386550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.386639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.386726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.386820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.386904] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.387006] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.387101] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.388193] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.398816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.399082] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.400120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.400208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.400291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.400372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.400453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.400532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.400611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.400688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.400767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.400857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.400934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.401011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.401088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.401165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.401242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.401319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.401394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.401469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.401544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.401629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.401715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.401798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.401904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.401986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.402067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.402146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.402225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.402303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.402378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.402457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.402538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.402617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.402695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.402774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.402872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.402957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.403041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.403122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.403203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.403279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.403357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.403432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.403523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.403602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.403683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.403762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.403877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.403970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.404050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.404131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.404213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.404290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.404373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.404454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.404531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.404609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.404688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.404766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.404855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.404935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.405013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.405092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.405169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.405245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.405323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.405400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.405479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.405557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.405638] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.405761] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.405766] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61943, lrc_seqno=61943, guc_id=0, flags=0x73 in no process [-1]
<7> [515.405769] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.407164] ------------[ cut here ]------------
<4> [515.407165] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.407167] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [515.407254] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.407325] [last unloaded: snd_hda_intel]
<4> [515.407329] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.407332] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.407334] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.407336] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.407342] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.407421] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.407423] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.407425] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.407427] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.407428] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.407429] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.407431] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.407432] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.407434] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.407435] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.407436] PKRU: 55555554
<4> [515.407438] Call Trace:
<4> [515.407439] <TASK>
<4> [515.407443] ? lock_acquire+0xb0/0x300
<4> [515.407450] ? lock_release+0xd0/0x2b0
<4> [515.407455] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.407461] process_one_work+0x239/0x740
<4> [515.407469] worker_thread+0x200/0x3f0
<4> [515.407472] ? __pfx_worker_thread+0x10/0x10
<4> [515.407475] kthread+0x10d/0x150
<4> [515.407477] ? __pfx_kthread+0x10/0x10
<4> [515.407480] ret_from_fork+0x3bd/0x470
<4> [515.407483] ? __pfx_kthread+0x10/0x10
<4> [515.407485] ret_from_fork_asm+0x1a/0x30
<4> [515.407493] </TASK>
<4> [515.407495] irq event stamp: 1898577
<4> [515.407496] hardirqs last enabled at (1898583): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.407498] hardirqs last disabled at (1898588): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.407500] softirqs last enabled at (1897820): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.407503] softirqs last disabled at (1897709): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.407506] ---[ end trace 0000000000000000 ]---
<7> [515.407906] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.408020] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [515.408124] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61998, lrc_seqno=61998, guc_id=0, flags=0x73 in no process [-1]
<7> [515.408127] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.408196] ------------[ cut here ]------------
<4> [515.408197] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.408198] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [515.408276] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.408342] [last unloaded: snd_hda_intel]
<4> [515.408345] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.408348] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.408349] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.408351] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.408356] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.408432] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.408433] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.408435] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.408437] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.408438] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.408439] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.408440] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.408442] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.408443] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.408445] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.408446] PKRU: 55555554
<4> [515.408447] Call Trace:
<4> [515.408448] <TASK>
<4> [515.408452] ? lock_acquire+0xb0/0x300
<4> [515.408457] ? lock_release+0xd0/0x2b0
<4> [515.408462] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.408468] process_one_work+0x239/0x740
<4> [515.408474] worker_thread+0x200/0x3f0
<4> [515.408477] ? __pfx_worker_thread+0x10/0x10
<4> [515.408480] kthread+0x10d/0x150
<4> [515.408482] ? __pfx_kthread+0x10/0x10
<4> [515.408485] ret_from_fork+0x3bd/0x470
<4> [515.408488] ? __pfx_kthread+0x10/0x10
<4> [515.408490] ret_from_fork_asm+0x1a/0x30
<4> [515.408498] </TASK>
<4> [515.408499] irq event stamp: 1900581
<4> [515.408500] hardirqs last enabled at (1900587): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.408502] hardirqs last disabled at (1900592): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.408504] softirqs last enabled at (1897820): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.408507] softirqs last disabled at (1897709): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.408509] ---[ end trace 0000000000000000 ]---
<6> [515.408511] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.408591] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.408597] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.408950] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.409175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.409277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.409376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.409469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.409560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.409647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.409733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.409829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.409913] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.410016] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.410112] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.411151] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.421815] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.422086] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.423095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.423185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.423268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.423351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.423433] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.423514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.423596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.423678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.423757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.423844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.423920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.423996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.424073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.424149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.424224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.424302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.424378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.424452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.424527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.424615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.424700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.424783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.424879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.424961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.425042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.425120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.425197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.425275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.425352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.425432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.425513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.425592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.425670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.425750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.425867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.425951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.426036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.426116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.426196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.426273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.426351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.426428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.426504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.426603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.426757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.426902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.427001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.427084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.427165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.427247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.427330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.427409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.427488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.427565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.427642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.427721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.427799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.427892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.427968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.428047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.428124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.428201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.428276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.428351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.428427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.428504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.428579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.428654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.428731] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.428861] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.428866] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61998, lrc_seqno=61998, guc_id=0, flags=0x73 in no process [-1]
<7> [515.428868] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.428937] ------------[ cut here ]------------
<4> [515.428939] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.428940] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#7: kworker/u64:45/4176
<4> [515.429020] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.429087] [last unloaded: snd_hda_intel]
<4> [515.429091] CPU: 7 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.429094] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.429095] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.429097] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.429103] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.429179] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.429181] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.429183] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.429185] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.429186] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.429188] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.429189] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.429190] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [515.429192] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.429193] CR2: 0000727e7544b098 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [515.429195] PKRU: 55555554
<4> [515.429196] Call Trace:
<4> [515.429197] <TASK>
<4> [515.429201] ? lock_acquire+0xb0/0x300
<4> [515.429207] ? lock_release+0xd0/0x2b0
<4> [515.429212] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.429218] process_one_work+0x239/0x740
<4> [515.429225] worker_thread+0x200/0x3f0
<4> [515.429228] ? __pfx_worker_thread+0x10/0x10
<4> [515.429230] kthread+0x10d/0x150
<4> [515.429233] ? __pfx_kthread+0x10/0x10
<4> [515.429236] ret_from_fork+0x3bd/0x470
<4> [515.429238] ? __pfx_kthread+0x10/0x10
<4> [515.429241] ret_from_fork_asm+0x1a/0x30
<4> [515.429248] </TASK>
<4> [515.429250] irq event stamp: 429875
<4> [515.429251] hardirqs last enabled at (429881): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.429253] hardirqs last disabled at (429886): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.429255] softirqs last enabled at (428886): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.429258] softirqs last disabled at (428879): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.429261] ---[ end trace 0000000000000000 ]---
<6> [515.429263] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.429339] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.429346] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.430250] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.430365] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.430478] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.430702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.430810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.430911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.431007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.431099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.431187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.431275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.431365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.431447] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.431549] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.431643] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.432683] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.443413] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.443675] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.444732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.444824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.444907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.444989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.445068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.445148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.445225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.445302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.445379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.445456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.445533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.445609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.445685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.445762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.445847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.445923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.446000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.446076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.446151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.446239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.446324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.446407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.446488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.446572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.446654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.446737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.446822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.446902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.446980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.447059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.447140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.447218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.447298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.447380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.447466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.447552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.447638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.447719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.447802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.447894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.447974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.448055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.448133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.448213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.448292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.448371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.448449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.448527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.448604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.448682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.448762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.448847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.448925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.449002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.449078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.449156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.449234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.449309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.449385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.449463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.449539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.449616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.449692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.449768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.449854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.449933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.450009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.450086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.450165] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.450287] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.450292] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=61998, lrc_seqno=61998, guc_id=0, flags=0x73 in no process [-1]
<7> [515.450295] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.450364] ------------[ cut here ]------------
<4> [515.450365] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.450367] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:43/4174
<4> [515.450446] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.450515] [last unloaded: snd_hda_intel]
<4> [515.450518] CPU: 2 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.450521] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.450523] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.450524] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.450530] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.450607] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.450609] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [515.450611] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.450612] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.450614] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.450615] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.450616] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.450617] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.450619] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.450621] CR2: 00007bb19a83a058 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [515.450622] PKRU: 55555554
<4> [515.450623] Call Trace:
<4> [515.450625] <TASK>
<4> [515.450629] ? lock_acquire+0xb0/0x300
<4> [515.450635] ? lock_release+0xd0/0x2b0
<4> [515.450640] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.450646] process_one_work+0x239/0x740
<4> [515.450653] worker_thread+0x200/0x3f0
<4> [515.450656] ? __pfx_worker_thread+0x10/0x10
<4> [515.450659] kthread+0x10d/0x150
<4> [515.450661] ? __pfx_kthread+0x10/0x10
<4> [515.450664] ret_from_fork+0x3bd/0x470
<4> [515.450666] ? __pfx_kthread+0x10/0x10
<4> [515.450669] ret_from_fork_asm+0x1a/0x30
<4> [515.450677] </TASK>
<4> [515.450678] irq event stamp: 469201
<4> [515.450679] hardirqs last enabled at (469207): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.450682] hardirqs last disabled at (469212): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.450683] softirqs last enabled at (468172): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.450686] softirqs last disabled at (468165): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.450689] ---[ end trace 0000000000000000 ]---
<5> [515.451876] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62053, lrc_seqno=62053, guc_id=0, flags=0x73 in no process [-1]
<7> [515.451881] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.451967] ------------[ cut here ]------------
<4> [515.451968] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.451970] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [515.452055] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.452122] [last unloaded: snd_hda_intel]
<4> [515.452125] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.452128] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.452129] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.452131] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.452137] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.452233] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.452235] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.452237] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.452239] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.452240] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.452241] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.452242] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.452244] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [515.452245] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.452247] CR2: 0000727e7e545c60 CR3: 000000010f264003 CR4: 0000000000f72ef0
<4> [515.452248] PKRU: 55555554
<4> [515.452249] Call Trace:
<4> [515.452251] <TASK>
<4> [515.452254] ? lock_acquire+0xb0/0x300
<4> [515.452260] ? lock_release+0xd0/0x2b0
<4> [515.452265] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.452271] process_one_work+0x239/0x740
<4> [515.452278] worker_thread+0x200/0x3f0
<4> [515.452281] ? __pfx_worker_thread+0x10/0x10
<4> [515.452284] kthread+0x10d/0x150
<4> [515.452286] ? __pfx_kthread+0x10/0x10
<4> [515.452289] ret_from_fork+0x3bd/0x470
<4> [515.452291] ? __pfx_kthread+0x10/0x10
<4> [515.452294] ret_from_fork_asm+0x1a/0x30
<4> [515.452302] </TASK>
<4> [515.452303] irq event stamp: 1903485
<4> [515.452304] hardirqs last enabled at (1903491): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.452306] hardirqs last disabled at (1903496): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.452308] softirqs last enabled at (1902412): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.452311] softirqs last disabled at (1902407): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.452314] ---[ end trace 0000000000000000 ]---
<6> [515.452316] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.452393] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.452399] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.452460] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.452578] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.452811] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.453030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.453135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.453235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.453339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.453428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.453514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.453601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.453689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.453772] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.453882] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.453977] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.455029] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.464820] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.465090] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.466141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.466233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.466318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.466411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.466493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.466575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.466655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.466734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.466819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.466902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.466980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.467055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.467131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.467206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.467281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.467355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.467429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.467504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.467579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.467667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.467752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.467843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.467922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.468002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.468081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.468158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.468233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.468308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.468383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.468461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.468540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.468617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.468695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.468773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.468871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.468956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.469041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.469121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.469201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.469277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.469352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.469426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.469501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.469577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.469655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.469735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.469822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.469902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.469980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.470058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.470138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.470215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.470291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.470371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.470446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.470523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.470599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.470674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.470748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.470833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.470910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.470988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.471064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.471140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.471217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.471293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.471371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.471449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.471529] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.471657] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.471662] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62053, lrc_seqno=62053, guc_id=0, flags=0x73 in no process [-1]
<7> [515.471664] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.471733] ------------[ cut here ]------------
<4> [515.471734] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.471736] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#0: kworker/u64:45/4176
<4> [515.471820] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.471888] [last unloaded: snd_hda_intel]
<4> [515.471891] CPU: 0 UID: 0 PID: 4176 Comm: kworker/u64:45 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.471894] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.471895] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.471897] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.471903] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.471980] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.471981] RSP: 0018:ffffc90005bb7ca0 EFLAGS: 00010246
<4> [515.471984] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.471985] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.471987] RBP: ffffc90005bb7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.471988] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.471989] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.471990] FS: 0000000000000000(0000) GS:ffff8888daa83000(0000) knlGS:0000000000000000
<4> [515.471992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.471993] CR2: 0000727e7d5e62f4 CR3: 000000010f264006 CR4: 0000000000f72ef0
<4> [515.471995] PKRU: 55555554
<4> [515.471996] Call Trace:
<4> [515.471997] <TASK>
<4> [515.472001] ? lock_acquire+0xb0/0x300
<4> [515.472007] ? lock_release+0xd0/0x2b0
<4> [515.472011] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.472018] process_one_work+0x239/0x740
<4> [515.472025] worker_thread+0x200/0x3f0
<4> [515.472028] ? __pfx_worker_thread+0x10/0x10
<4> [515.472031] kthread+0x10d/0x150
<4> [515.472033] ? __pfx_kthread+0x10/0x10
<4> [515.472036] ret_from_fork+0x3bd/0x470
<4> [515.472038] ? __pfx_kthread+0x10/0x10
<4> [515.472041] ret_from_fork_asm+0x1a/0x30
<4> [515.472049] </TASK>
<4> [515.472050] irq event stamp: 437683
<4> [515.472051] hardirqs last enabled at (437689): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.472053] hardirqs last disabled at (437694): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.472055] softirqs last enabled at (436606): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.472058] softirqs last disabled at (436601): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.472061] ---[ end trace 0000000000000000 ]---
<6> [515.472063] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [515.472142] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [515.472150] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [515.473066] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.473183] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [515.473298] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [515.473512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [515.473607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [515.473703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [515.473820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [515.473910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [515.473999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [515.474089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [515.474178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [515.474262] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [515.474363] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [515.474457] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [515.475545] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [515.485813] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [515.486081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [515.487148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [515.487239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [515.487325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [515.487411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [515.487495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [515.487575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [515.487654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [515.487733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [515.487818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [515.487899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [515.487977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [515.488055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [515.488131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [515.488208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [515.488285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [515.488361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [515.488438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [515.488515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [515.488591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [515.488678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [515.488764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [515.488860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [515.488941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [515.489024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [515.489105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [515.489186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [515.489266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [515.489345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [515.489423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [515.489504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [515.489583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [515.489662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [515.489741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [515.489827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [515.489917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [515.490002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [515.490087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [515.490172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [515.490253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [515.490332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [515.490411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [515.490488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [515.490565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [515.490647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [515.490728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [515.490813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [515.490892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [515.490969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [515.491045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [515.491122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [515.491202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [515.491278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [515.491355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [515.491430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [515.491505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [515.491582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [515.491659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [515.491733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [515.491815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [515.491895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [515.491971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [515.492048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [515.492125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [515.492200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [515.492277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [515.492354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [515.492429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [515.492505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [515.492584] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [515.492710] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [515.492715] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62053, lrc_seqno=62053, guc_id=0, flags=0x73 in no process [-1]
<7> [515.492717] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [515.492785] ------------[ cut here ]------------
<4> [515.492787] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [515.492788] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:13/2860
<4> [515.492884] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [515.492952] [last unloaded: snd_hda_intel]
<4> [515.492955] CPU: 2 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [515.492958] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [515.492960] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [515.492961] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [515.492967] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [515.493043] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [515.493045] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [515.493047] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [515.493049] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [515.493050] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [515.493051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [515.493053] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [515.493054] FS: 0000000000000000(0000) GS:ffff8888dab83000(0000) knlGS:0000000000000000
<4> [515.493056] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [515.493057] CR2: 00007bb19a83a058 CR3: 000000010f264001 CR4: 0000000000f72ef0
<4> [515.493058] PKRU: 55555554
<4> [515.493060] Call Trace:
<4> [515.493061] <TASK>
<4> [515.493065] ? lock_acquire+0xb0/0x300
<4> [515.493071] ? lock_release+0xd0/0x2b0
<4> [515.493076] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [515.493082] process_one_work+0x239/0x740
<4> [515.493089] worker_thread+0x200/0x3f0
<4> [515.493092] ? __pfx_worker_thread+0x10/0x10
<4> [515.493095] kthread+0x10d/0x150
<4> [515.493097] ? __pfx_kthread+0x10/0x10
<4> [515.493100] ret_from_fork+0x3bd/0x470
<4> [515.493102] ? __pfx_kthread+0x10/0x10
<4> [515.493105] ret_from_fork_asm+0x1a/0x30
<4> [515.493113] </TASK>
<4> [515.493114] irq event stamp: 1907477
<4> [515.493115] hardirqs last enabled at (1907483): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [515.493118] hardirqs last disabled at (1907488): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [515.493120] softirqs last enabled at (1906740): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.493123] softirqs last disabled at (1906735): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [515.493125] ---[ end trace 0000000000000000 ]---
<7> [515.494093] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [515.494207] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [517.053880] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62090, lrc_seqno=62090, guc_id=0, flags=0x73 in no process [-1]
<7> [517.053886] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [517.054014] ------------[ cut here ]------------
<4> [517.054015] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [517.054017] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:43/4174
<4> [517.054098] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [517.054168] [last unloaded: snd_hda_intel]
<4> [517.054172] CPU: 4 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [517.054175] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [517.054176] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [517.054177] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [517.054184] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [517.054262] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [517.054263] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [517.054265] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [517.054267] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [517.054268] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [517.054269] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [517.054270] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [517.054271] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [517.054273] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [517.054274] CR2: 00007df35c068018 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [517.054275] PKRU: 55555554
<4> [517.054276] Call Trace:
<4> [517.054277] <TASK>
<4> [517.054281] ? lock_acquire+0xb0/0x300
<4> [517.054287] ? lock_release+0xd0/0x2b0
<4> [517.054291] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [517.054297] process_one_work+0x239/0x740
<4> [517.054305] worker_thread+0x200/0x3f0
<4> [517.054308] ? __pfx_worker_thread+0x10/0x10
<4> [517.054310] kthread+0x10d/0x150
<4> [517.054313] ? __pfx_kthread+0x10/0x10
<4> [517.054315] ret_from_fork+0x3bd/0x470
<4> [517.054318] ? __pfx_kthread+0x10/0x10
<4> [517.054321] ret_from_fork_asm+0x1a/0x30
<4> [517.054328] </TASK>
<4> [517.054329] irq event stamp: 495509
<4> [517.054331] hardirqs last enabled at (495515): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [517.054333] hardirqs last disabled at (495520): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [517.054335] softirqs last enabled at (470120): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.054338] softirqs last disabled at (470115): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.054340] ---[ end trace 0000000000000000 ]---
<6> [517.054342] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [517.054419] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [517.054428] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [517.054612] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [517.054842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [517.054965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [517.055097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [517.055191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [517.055283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [517.055372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [517.055461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [517.055549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [517.055630] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [517.055730] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [517.055855] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [517.056928] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [517.067654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [517.067918] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [517.069093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [517.069186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [517.069270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [517.069349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [517.069426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [517.069504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [517.069580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [517.069655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [517.069731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [517.069809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [517.069891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [517.069975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [517.070049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [517.070125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [517.070201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [517.070274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [517.070348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [517.070422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [517.070495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [517.070577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [517.070658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [517.070740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [517.070825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [517.070911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [517.070992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [517.071066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [517.071142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [517.071217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [517.071292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [517.071370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [517.071448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [517.071525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [517.071601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [517.071677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [517.071759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [517.071851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [517.071940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [517.072027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [517.072112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [517.072196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [517.072279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [517.072361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [517.072442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [517.072524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [517.072607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [517.072691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [517.072773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [517.072868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [517.072949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [517.073031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [517.073116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [517.073196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [517.073278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [517.073358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [517.073438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [517.073519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [517.073601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [517.073682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [517.073764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [517.073857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [517.073941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [517.074028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [517.074114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [517.074200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [517.074290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [517.074379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [517.074467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [517.074557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [517.074651] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [517.075654] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [517.075661] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62090, lrc_seqno=62090, guc_id=0, flags=0x73 in no process [-1]
<7> [517.075665] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [517.075764] ------------[ cut here ]------------
<4> [517.075765] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [517.075767] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:43/4174
<4> [517.075876] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [517.075959] [last unloaded: snd_hda_intel]
<4> [517.075963] CPU: 4 UID: 0 PID: 4174 Comm: kworker/u64:43 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [517.075967] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [517.075969] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [517.075970] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [517.075978] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [517.076072] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [517.076074] RSP: 0018:ffffc90005bc7ca0 EFLAGS: 00010246
<4> [517.076077] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [517.076079] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [517.076081] RBP: ffffc90005bc7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [517.076082] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [517.076084] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [517.076086] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [517.076088] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [517.076089] CR2: 00007df35c068018 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [517.076091] PKRU: 55555554
<4> [517.076093] Call Trace:
<4> [517.076094] <TASK>
<4> [517.076099] ? lock_acquire+0xb0/0x300
<4> [517.076106] ? lock_release+0xd0/0x2b0
<4> [517.076112] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [517.076120] process_one_work+0x239/0x740
<4> [517.076128] worker_thread+0x200/0x3f0
<4> [517.076132] ? __pfx_worker_thread+0x10/0x10
<4> [517.076136] kthread+0x10d/0x150
<4> [517.076139] ? __pfx_kthread+0x10/0x10
<4> [517.076143] ret_from_fork+0x3bd/0x470
<4> [517.076146] ? __pfx_kthread+0x10/0x10
<4> [517.076149] ret_from_fork_asm+0x1a/0x30
<4> [517.076159] </TASK>
<4> [517.076161] irq event stamp: 498713
<4> [517.076162] hardirqs last enabled at (498719): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [517.076165] hardirqs last disabled at (498724): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [517.076167] softirqs last enabled at (497812): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.076171] softirqs last disabled at (497805): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.076174] ---[ end trace 0000000000000000 ]---
<6> [517.076176] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [517.076272] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [517.076280] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [517.076299] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [517.076434] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [517.076706] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [517.076980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [517.077097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [517.077209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [517.077311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [517.077411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [517.077504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [517.077599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [517.077696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [517.077786] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [517.077908] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [517.078007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [517.079048] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [517.088796] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [517.089061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [517.090126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [517.090215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [517.090298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [517.090380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [517.090460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [517.090541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [517.090620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [517.090701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [517.090779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [517.090875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [517.090952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [517.091029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [517.091105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [517.091181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [517.091257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [517.091330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [517.091404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [517.091478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [517.091551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [517.091639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [517.091725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [517.091828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [517.091909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [517.091991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [517.092071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [517.092149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [517.092225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [517.092303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [517.092386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [517.092464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [517.092544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [517.092624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [517.092705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [517.092786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [517.092891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [517.092978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [517.093064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [517.093146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [517.093227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [517.093305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [517.093382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [517.093459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [517.093533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [517.093613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [517.093692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [517.093771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [517.093865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [517.093947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [517.094023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [517.094101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [517.094181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [517.094258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [517.094336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [517.094413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [517.094489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [517.094567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [517.094647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [517.094725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [517.094808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [517.094891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [517.094968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [517.095045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [517.095121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [517.095197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [517.095274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [517.095350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [517.095425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [517.095499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [517.095577] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [517.095698] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [517.095703] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62090, lrc_seqno=62090, guc_id=0, flags=0x73 in no process [-1]
<7> [517.095705] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [517.095774] ------------[ cut here ]------------
<4> [517.095775] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [517.095776] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:13/2860
<4> [517.095871] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [517.095940] [last unloaded: snd_hda_intel]
<4> [517.095944] CPU: 1 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [517.095947] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [517.095949] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [517.095950] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [517.095956] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [517.096033] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [517.096035] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [517.096037] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [517.096039] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [517.096040] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [517.096041] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [517.096043] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [517.096044] FS: 0000000000000000(0000) GS:ffff8888dab03000(0000) knlGS:0000000000000000
<4> [517.096046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [517.096047] CR2: 0000727e7e545c60 CR3: 000000000364a004 CR4: 0000000000f72ef0
<4> [517.096049] PKRU: 55555554
<4> [517.096050] Call Trace:
<4> [517.096051] <TASK>
<4> [517.096055] ? lock_acquire+0xb0/0x300
<4> [517.096061] ? lock_release+0xd0/0x2b0
<4> [517.096066] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [517.096072] process_one_work+0x239/0x740
<4> [517.096079] worker_thread+0x200/0x3f0
<4> [517.096082] ? __pfx_worker_thread+0x10/0x10
<4> [517.096085] kthread+0x10d/0x150
<4> [517.096088] ? __pfx_kthread+0x10/0x10
<4> [517.096091] ret_from_fork+0x3bd/0x470
<4> [517.096093] ? __pfx_kthread+0x10/0x10
<4> [517.096096] ret_from_fork_asm+0x1a/0x30
<4> [517.096104] </TASK>
<4> [517.096105] irq event stamp: 2263105
<4> [517.096106] hardirqs last enabled at (2263111): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [517.096109] hardirqs last disabled at (2263116): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [517.096110] softirqs last enabled at (2262028): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.096113] softirqs last disabled at (2262021): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [517.096116] ---[ end trace 0000000000000000 ]---
<7> [517.097393] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [517.097507] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [519.160005] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62092, lrc_seqno=62092, guc_id=0, flags=0x73 in no process [-1]
<7> [519.160012] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [519.160102] ------------[ cut here ]------------
<4> [519.160103] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [519.160105] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:9/1976
<4> [519.160181] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [519.160246] [last unloaded: snd_hda_intel]
<4> [519.160250] CPU: 4 UID: 0 PID: 1976 Comm: kworker/u64:9 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [519.160252] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [519.160254] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [519.160255] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [519.160261] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [519.160334] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [519.160336] RSP: 0018:ffffc90003b4fca0 EFLAGS: 00010246
<4> [519.160338] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [519.160339] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [519.160341] RBP: ffffc90003b4fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [519.160342] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [519.160343] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [519.160344] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [519.160345] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [519.160347] CR2: 00007f623e60f000 CR3: 000000000364a005 CR4: 0000000000f72ef0
<4> [519.160348] PKRU: 55555554
<4> [519.160349] Call Trace:
<4> [519.160350] <TASK>
<4> [519.160354] ? lock_acquire+0xb0/0x300
<4> [519.160359] ? lock_release+0xd0/0x2b0
<4> [519.160364] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [519.160369] process_one_work+0x239/0x740
<4> [519.160376] worker_thread+0x200/0x3f0
<4> [519.160379] ? __pfx_worker_thread+0x10/0x10
<4> [519.160381] kthread+0x10d/0x150
<4> [519.160383] ? __pfx_kthread+0x10/0x10
<4> [519.160386] ret_from_fork+0x3bd/0x470
<4> [519.160389] ? __pfx_kthread+0x10/0x10
<4> [519.160391] ret_from_fork_asm+0x1a/0x30
<4> [519.160399] </TASK>
<4> [519.160400] irq event stamp: 731753
<4> [519.160401] hardirqs last enabled at (731759): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [519.160403] hardirqs last disabled at (731764): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [519.160405] softirqs last enabled at (496868): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.160408] softirqs last disabled at (496863): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.160410] ---[ end trace 0000000000000000 ]---
<6> [519.160412] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [519.160484] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [519.160491] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [519.160543] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [519.160762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [519.160962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [519.161124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [519.161251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [519.161358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [519.161445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [519.161529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [519.161613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [519.161693] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [519.161796] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [519.161907] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [519.162946] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [519.173720] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [519.174014] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [519.175255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [519.175338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [519.175414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [519.175490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [519.175564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [519.175635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [519.175706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [519.175782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [519.175860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [519.175938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [519.176008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [519.176080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [519.176152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [519.176223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [519.176295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [519.176363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [519.176432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [519.176500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [519.176569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [519.176649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [519.176727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [519.176824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [519.176905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [519.176981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [519.177053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [519.177123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [519.177195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [519.177265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [519.177335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [519.177407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [519.177480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [519.177550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [519.177623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [519.177695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [519.177777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [519.177861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [519.177943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [519.178017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [519.178090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [519.178160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [519.178232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [519.178302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [519.178372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [519.178442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [519.178513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [519.178584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [519.178655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [519.178725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [519.178807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [519.178887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [519.178963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [519.179034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [519.179104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [519.179174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [519.179241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [519.179311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [519.179381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [519.179449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [519.179517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [519.179588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [519.179658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [519.179728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [519.179836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [519.179967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [519.180069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [519.180146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [519.180219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [519.180291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [519.180366] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [519.180484] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [519.180489] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62092, lrc_seqno=62092, guc_id=0, flags=0x73 in no process [-1]
<7> [519.180491] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [519.180554] ------------[ cut here ]------------
<4> [519.180555] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [519.180557] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#9: kworker/u64:13/2860
<4> [519.180629] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [519.180692] [last unloaded: snd_hda_intel]
<4> [519.180695] CPU: 9 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [519.180698] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [519.180699] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [519.180700] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [519.180706] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [519.180782] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [519.180784] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [519.180786] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [519.180788] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [519.180789] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [519.180790] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [519.180791] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [519.180792] FS: 0000000000000000(0000) GS:ffff8888daf03000(0000) knlGS:0000000000000000
<4> [519.180794] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [519.180795] CR2: 00007f623e60f000 CR3: 000000000364a002 CR4: 0000000000f72ef0
<4> [519.180797] PKRU: 55555554
<4> [519.180798] Call Trace:
<4> [519.180799] <TASK>
<4> [519.180804] ? lock_acquire+0xb0/0x300
<4> [519.180809] ? lock_release+0xd0/0x2b0
<4> [519.180815] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [519.180821] process_one_work+0x239/0x740
<4> [519.180828] worker_thread+0x200/0x3f0
<4> [519.180831] ? __pfx_worker_thread+0x10/0x10
<4> [519.180834] kthread+0x10d/0x150
<4> [519.180837] ? __pfx_kthread+0x10/0x10
<4> [519.180840] ret_from_fork+0x3bd/0x470
<4> [519.180843] ? __pfx_kthread+0x10/0x10
<4> [519.180846] ret_from_fork_asm+0x1a/0x30
<4> [519.180853] </TASK>
<4> [519.180855] irq event stamp: 2296355
<4> [519.180856] hardirqs last enabled at (2296361): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [519.180859] hardirqs last disabled at (2296366): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [519.180860] softirqs last enabled at (2295480): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.180863] softirqs last disabled at (2295473): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.180866] ---[ end trace 0000000000000000 ]---
<6> [519.180868] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [519.180944] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [519.180950] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [519.181361] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [519.181570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [519.181663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [519.181757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [519.181862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [519.181948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [519.182032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [519.182116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [519.182203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [519.182285] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [519.182386] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [519.182480] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [519.183517] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [519.193777] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [519.194042] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [519.195104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [519.195191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [519.195273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [519.195355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [519.195435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [519.195515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [519.195592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [519.195669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [519.195744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [519.195835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [519.195912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [519.195989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [519.196066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [519.196143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [519.196218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [519.196296] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [519.196372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [519.196447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [519.196523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [519.196610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [519.196696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [519.196786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [519.196869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [519.196953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [519.197034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [519.197112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [519.197189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [519.197266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [519.197343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [519.197423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [519.197503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [519.197582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [519.197660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [519.197739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [519.197833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [519.197917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [519.198000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [519.198081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [519.198163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [519.198241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [519.198319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [519.198395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [519.198472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [519.198550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [519.198629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [519.198707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [519.198790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [519.198866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [519.198943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [519.199020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [519.199100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [519.199177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [519.199253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [519.199329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [519.199404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [519.199481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [519.199558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [519.199633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [519.199709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [519.199791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [519.199873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [519.199953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [519.200032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [519.200108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [519.200185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [519.200262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [519.200337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [519.200413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [519.200492] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [519.200611] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [519.200615] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=62092, lrc_seqno=62092, guc_id=0, flags=0x73 in no process [-1]
<7> [519.200618] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [519.200685] ------------[ cut here ]------------
<4> [519.200686] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [519.200688] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#9: kworker/u64:13/2860
<4> [519.201217] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp hid_generic coretemp asus_nb_wmi asus_wmi sparse_keymap cmdlinepart mei_pxp mei_hdcp binfmt_misc spi_nor wmi_bmof mtd kvm_intel usbhid r8169 kvm irqbypass hid aesni_intel gf128mul rapl realtek intel_cstate phy_package snd_intel_dspcfg snd_hda_codec snd_hda_core intel_pmc_core snd_hwdep nls_iso8859_1 video snd_pcm pmt_telemetry pmt_discovery i2c_i801 pmt_class snd_timer snd i2c_mux intel_pmc_ssram_telemetry mei_me acpi_pad spi_intel_pci i2c_smbus acpi_tad soundcore pinctrl_alderlake idma64 spi_intel wmi mei intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [519.201282] [last unloaded: snd_hda_intel]
<4> [519.201285] CPU: 9 UID: 0 PID: 2860 Comm: kworker/u64:13 Tainted: G S U W 7.1.0-rc5-lgci-xe-xe-5137-5390f2273d45bb259-debug+ #1 PREEMPT(lazy)
<4> [519.201288] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [519.201289] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [519.201291] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [519.201297] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [519.201378] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 e0 ed 3c e1 48 89 c6 48 8d 3d f6 88 d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [519.201380] RSP: 0018:ffffc900023a7ca0 EFLAGS: 00010246
<4> [519.201382] RAX: ffffffffa12bc25a RBX: 0000000000000000 RCX: 0000000000000000
<4> [519.201384] RDX: ffff888104285410 RSI: ffffffffa12bc25a RDI: ffffffffa0c04020
<4> [519.201385] RBP: ffffc900023a7db0 R08: 0000000000000000 R09: 0000000000000000
<4> [519.201386] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [519.201387] R13: ffff888104285410 R14: ffff88814355a018 R15: 00000000ffffffc2
<4> [519.201389] FS: 0000000000000000(0000) GS:ffff8888daf03000(0000) knlGS:0000000000000000
<4> [519.201390] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [519.201392] CR2: 00007f623e60f000 CR3: 000000000364a002 CR4: 0000000000f72ef0
<4> [519.201393] PKRU: 55555554
<4> [519.201395] Call Trace:
<4> [519.201396] <TASK>
<4> [519.201399] ? lock_acquire+0xb0/0x300
<4> [519.201405] ? lock_release+0xd0/0x2b0
<4> [519.201410] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [519.201416] process_one_work+0x239/0x740
<4> [519.201422] worker_thread+0x200/0x3f0
<4> [519.201425] ? __pfx_worker_thread+0x10/0x10
<4> [519.201428] kthread+0x10d/0x150
<4> [519.201430] ? __pfx_kthread+0x10/0x10
<4> [519.201434] ret_from_fork+0x3bd/0x470
<4> [519.201436] ? __pfx_kthread+0x10/0x10
<4> [519.201439] ret_from_fork_asm+0x1a/0x30
<4> [519.201446] </TASK>
<4> [519.201447] irq event stamp: 2299509
<4> [519.201449] hardirqs last enabled at (2299515): [<ffffffff814ac629>] __up_console_sem+0x79/0xa0
<4> [519.201451] hardirqs last disabled at (2299520): [<ffffffff814ac60e>] __up_console_sem+0x5e/0xa0
<4> [519.201453] softirqs last enabled at (2298528): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.201456] softirqs last disabled at (2298521): [<ffffffff813d333b>] __irq_exit_rcu+0xdb/0x1c0
<4> [519.201458] ---[ end trace 0000000000000000 ]---
<7> [519.202714] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [519.202842] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [520.719222] [IGT] xe_pat: finished subtest pt-caching, SUCCESS
<7> [520.735691] xe 0000:03:00.0: [drm:drm_client_dev_restore] fbdev: ret=0
<6> [520.736244] [IGT] xe_pat: exiting, ret=0
<6> [520.751349] Console: switching to colour frame buffer device 240x67
|