Result: 368 Warning(s)
i915_display_info3 igt_runner3 results3.json results3-xe-load.json guc_logs3.tar i915_display_info_post_exec3 serial_data3 boot3 dmesg3
| Detail | Value |
|---|---|
| Duration | unknown |
| Hostname |
shard-bmg-4 |
| Igt-Version |
IGT-Version: 2.4-g8aca0b828 (x86_64) (Linux: 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ x86_64) |
| Out |
Starting dynamic subtest: pipe-C-DP-2-size-128 Dynamic subtest pipe-C-DP-2-size-128: SUCCESS (4.198s) This test caused an abort condition: Kernel badly tainted (0x4244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened. |
| Err |
Starting dynamic subtest: pipe-C-DP-2-size-128 Dynamic subtest pipe-C-DP-2-size-128: SUCCESS (4.198s) |
| Dmesg |
<6> [242.498588] [IGT] kms_plane_cursor: starting dynamic subtest pipe-C-DP-2-size-128
<7> [242.500089] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:556]
<7> [242.501360] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:558]
<7> [242.515814] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:560]
<6> [242.515821] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x000000000060b000
FaultType: 0
AccessType: 1
FaultLevel: 0
EngineClass: 3 bcs
EngineInstance: 8
<6> [242.515828] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<6> [242.515904] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<6> [242.516486] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [242.516497] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38026, lrc_seqno=38026, guc_id=0, flags=0x73 in no process [-1]
<7> [242.516501] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.516601] ------------[ cut here ]------------
<4> [242.516603] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.516605] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [242.516720] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.516820]
<4> [242.516824] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.516827] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [L]=SOFTLOCKUP
<4> [242.516829] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.516830] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.516837] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.516930] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.516932] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [242.516935] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.516936] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.516938] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.516939] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.516940] R13: ffff888103f8e590 R14: ffff888108668000 R15: 00000000fffffffb
<4> [242.516942] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [242.516943] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.516945] CR2: 00007fe398449000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.516947] PKRU: 55555554
<4> [242.516948] Call Trace:
<4> [242.516949] <TASK>
<4> [242.516954] ? lock_acquire+0x60/0x300
<4> [242.516960] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [242.516966] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.516973] process_one_work+0x239/0x740
<4> [242.516981] worker_thread+0x200/0x3f0
<4> [242.516985] ? __pfx_worker_thread+0x10/0x10
<4> [242.516988] kthread+0x10d/0x150
<4> [242.516991] ? __pfx_kthread+0x10/0x10
<4> [242.516994] ret_from_fork+0x3bd/0x470
<4> [242.516997] ? __pfx_kthread+0x10/0x10
<4> [242.517000] ret_from_fork_asm+0x1a/0x30
<4> [242.517010] </TASK>
<4> [242.517011] irq event stamp: 1294241
<4> [242.517014] hardirqs last enabled at (1294247): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.517019] hardirqs last disabled at (1294252): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.517021] softirqs last enabled at (1293498): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.517025] softirqs last disabled at (1293491): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.517027] ---[ end trace 0000000000000000 ]---
<6> [242.517030] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.517122] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.517133] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.517213] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.517440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.517554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.517878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.517992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.518104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.518220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.518363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.518478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.518592] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.518761] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.518887] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.520002] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.530682] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.530987] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.532163] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.532302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.532432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.532557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.532691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.532814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.532936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.533053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.533168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.533280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.533392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.533505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.533616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.533749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.533856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.533959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.534064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.534162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.534260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.534373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.534485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.534591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.534699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.534800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.534899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.534995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.535087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.535178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.535268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.535364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.535463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.535555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.535648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.535746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.535836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.535925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.536012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.536094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.536177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.536256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.536335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.536412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.536491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.536571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.536649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.536739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.536818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.536895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.536971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.537048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.537126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.537203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.537280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.537356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.537432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.537510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.537587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.537662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.537761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.537841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.537917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.537999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.538075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.538150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.538226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.538301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.538376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.538450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.538527] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.538715] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.538720] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38026, lrc_seqno=38026, guc_id=0, flags=0x73 in no process [-1]
<7> [242.538723] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.538791] ------------[ cut here ]------------
<4> [242.538792] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.538794] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [242.538873] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.538940]
<4> [242.538942] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.538946] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.538947] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.538948] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.538954] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.539030] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.539032] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [242.539034] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.539036] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.539037] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.539038] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.539039] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.539041] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [242.539042] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.539044] CR2: 00007fe398449000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.539045] PKRU: 55555554
<4> [242.539046] Call Trace:
<4> [242.539047] <TASK>
<4> [242.539051] ? lock_acquire+0x60/0x300
<4> [242.539056] ? lock_release+0xd0/0x2b0
<4> [242.539061] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.539067] process_one_work+0x239/0x740
<4> [242.539074] worker_thread+0x200/0x3f0
<4> [242.539077] ? __pfx_worker_thread+0x10/0x10
<4> [242.539080] kthread+0x10d/0x150
<4> [242.539082] ? __pfx_kthread+0x10/0x10
<4> [242.539085] ret_from_fork+0x3bd/0x470
<4> [242.539088] ? __pfx_kthread+0x10/0x10
<4> [242.539091] ret_from_fork_asm+0x1a/0x30
<4> [242.539099] </TASK>
<4> [242.539100] irq event stamp: 1297537
<4> [242.539101] hardirqs last enabled at (1297543): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.539104] hardirqs last disabled at (1297548): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.539105] softirqs last enabled at (1296728): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.539109] softirqs last disabled at (1296721): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.539111] ---[ end trace 0000000000000000 ]---
<6> [242.539113] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.539189] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.539195] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.539462] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.539679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.539770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.539864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.539954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.540040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.540126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.540212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.540300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.540382] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.540483] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.540577] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.541604] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.552340] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.552598] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.553641] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.553743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.553826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.553907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.553984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.554064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.554142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.554220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.554295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.554371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.554446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.554523] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.554600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.554681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.554760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.554839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.554916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.554993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.555068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.555152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.555237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.555320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.555400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.555479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.555559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.555637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.555726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.555802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.555879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.555961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.556043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.556120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.556198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.556275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.556357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.556440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.556523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.556602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.556685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.556761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.556841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.556918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.556992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.557068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.557159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.557246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.557324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.557402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.557478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.557559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.557640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.557728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.557806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.557881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.557955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.558030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.558105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.558178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.558252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.558329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.558405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.558482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.558559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.558632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.558720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.558797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.558872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.558947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.559024] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.559293] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.559297] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38026, lrc_seqno=38026, guc_id=0, flags=0x73 in no process [-1]
<7> [242.559300] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.559368] ------------[ cut here ]------------
<4> [242.559369] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.559370] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [242.559450] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.559661]
<4> [242.559664] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.559667] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.559668] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.559777] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.559783] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.559866] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.559868] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [242.559870] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.559872] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.559873] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.559874] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.559875] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.559877] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [242.559878] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.559880] CR2: 00007fe398449000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.559881] PKRU: 55555554
<4> [242.559882] Call Trace:
<4> [242.559883] <TASK>
<4> [242.559887] ? lock_acquire+0x60/0x300
<4> [242.559892] ? lock_release+0xd0/0x2b0
<4> [242.559897] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.559903] process_one_work+0x239/0x740
<4> [242.559910] worker_thread+0x200/0x3f0
<4> [242.559913] ? __pfx_worker_thread+0x10/0x10
<4> [242.559916] kthread+0x10d/0x150
<4> [242.559918] ? __pfx_kthread+0x10/0x10
<4> [242.559921] ret_from_fork+0x3bd/0x470
<4> [242.559923] ? __pfx_kthread+0x10/0x10
<4> [242.559926] ret_from_fork_asm+0x1a/0x30
<4> [242.559933] </TASK>
<4> [242.559935] irq event stamp: 1300715
<4> [242.559936] hardirqs last enabled at (1300721): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.559938] hardirqs last disabled at (1300726): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.559940] softirqs last enabled at (1299376): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.559943] softirqs last disabled at (1299367): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.559945] ---[ end trace 0000000000000000 ]---
<7> [242.561082] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.561199] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [242.565076] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38038, lrc_seqno=38038, guc_id=0, flags=0x73 in no process [-1]
<7> [242.565079] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.565154] ------------[ cut here ]------------
<4> [242.565156] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.565157] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:8/983
<4> [242.565237] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.565302]
<4> [242.565304] CPU: 11 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.565307] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.565308] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.565310] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.565315] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.565392] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.565393] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [242.565395] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.565397] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.565398] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.565399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.565400] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.565402] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.565403] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.565404] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.565406] PKRU: 55555554
<4> [242.565407] Call Trace:
<4> [242.565408] <TASK>
<4> [242.565412] ? lock_acquire+0x60/0x300
<4> [242.565417] ? lock_release+0xd0/0x2b0
<4> [242.565422] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.565428] process_one_work+0x239/0x740
<4> [242.565434] worker_thread+0x200/0x3f0
<4> [242.565437] ? __pfx_worker_thread+0x10/0x10
<4> [242.565440] kthread+0x10d/0x150
<4> [242.565442] ? __pfx_kthread+0x10/0x10
<4> [242.565445] ret_from_fork+0x3bd/0x470
<4> [242.565448] ? __pfx_kthread+0x10/0x10
<4> [242.565450] ret_from_fork_asm+0x1a/0x30
<4> [242.565458] </TASK>
<4> [242.565459] irq event stamp: 1302109
<4> [242.565460] hardirqs last enabled at (1302115): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.565462] hardirqs last disabled at (1302120): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.565464] softirqs last enabled at (1299376): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.565467] softirqs last disabled at (1299367): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.565469] ---[ end trace 0000000000000000 ]---
<6> [242.565471] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.565546] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.565552] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.565899] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.566110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.566206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.566303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.566396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.566486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.566573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.566662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.566769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.566853] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.566965] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.567059] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.568092] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.578678] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.578935] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.579959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.580044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.580123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.580204] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.580284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.580361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.580438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.580512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.580587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.580661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.580771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.580849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.580923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.580997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.581072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.581146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.581222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.581299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.581374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.581460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.581544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.581625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.581716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.581798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.581877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.581955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.582033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.582109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.582186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.582266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.582347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.582424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.582504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.582584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.582678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.582769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.582856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.582938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.583018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.583097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.583176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.583254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.583333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.583414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.583495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.583576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.583655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.583746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.583823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.583900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.583980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.584056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.584134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.584210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.584285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.584361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.584438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.584513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.584588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.584666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.584758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.584835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.584911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.584986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.585063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.585138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.585211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.585286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.585363] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.585613] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.585618] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38038, lrc_seqno=38038, guc_id=0, flags=0x73 in no process [-1]
<7> [242.585620] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.585701] ------------[ cut here ]------------
<4> [242.585702] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.585704] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:8/983
<4> [242.585782] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.585846]
<4> [242.585848] CPU: 11 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.585851] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.585852] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.585854] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.585859] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.585935] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.585937] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [242.585939] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.585941] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.585942] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.585943] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.585944] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.585946] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.585947] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.585949] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.585950] PKRU: 55555554
<4> [242.585951] Call Trace:
<4> [242.585952] <TASK>
<4> [242.585956] ? lock_acquire+0x60/0x300
<4> [242.585961] ? lock_release+0xd0/0x2b0
<4> [242.585966] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.585972] process_one_work+0x239/0x740
<4> [242.585979] worker_thread+0x200/0x3f0
<4> [242.585982] ? __pfx_worker_thread+0x10/0x10
<4> [242.585985] kthread+0x10d/0x150
<4> [242.585987] ? __pfx_kthread+0x10/0x10
<4> [242.585990] ret_from_fork+0x3bd/0x470
<4> [242.585992] ? __pfx_kthread+0x10/0x10
<4> [242.585995] ret_from_fork_asm+0x1a/0x30
<4> [242.586003] </TASK>
<4> [242.586004] irq event stamp: 1305327
<4> [242.586005] hardirqs last enabled at (1305333): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.586007] hardirqs last disabled at (1305338): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.586008] softirqs last enabled at (1304566): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.586011] softirqs last disabled at (1304559): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.586014] ---[ end trace 0000000000000000 ]---
<6> [242.586015] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.586095] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.586102] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.587614] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.587748] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.587951] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.588185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.588295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.588402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.588500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.588596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.588700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.588795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.588891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.588981] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.589090] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.589192] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.590233] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.600678] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.600941] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.601935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.602028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.602115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.602199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.602282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.602366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.602449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.602532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.602614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.602702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.602784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.602865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.602946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.603027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.603108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.603207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.603290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.603371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.603454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.603547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.603637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.603735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.603821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.603907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.603992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.604078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.604164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.604249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.604332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.604418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.604504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.604589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.604679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.604765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.604856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.604946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.605036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.605123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.605208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.605292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.605375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.605457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.605540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.605623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.605715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.605798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.605879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.605965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.606047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.606129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.606220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.606305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.606393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.606476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.606558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.606644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.606734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.606816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.606896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.606980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.607062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.607143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.607225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.607305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.607387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.607469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.607550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.607630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.607724] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.607854] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.607859] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38038, lrc_seqno=38038, guc_id=0, flags=0x73 in no process [-1]
<7> [242.607861] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.607933] ------------[ cut here ]------------
<4> [242.607935] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.607936] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [242.608020] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.608087]
<4> [242.608089] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.608093] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.608094] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.608095] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.608102] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.608183] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.608185] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.608188] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.608189] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.608190] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.608192] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.608193] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.608194] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [242.608196] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.608197] CR2: 00007fe38c095208 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.608199] PKRU: 55555554
<4> [242.608200] Call Trace:
<4> [242.608201] <TASK>
<4> [242.608205] ? lock_acquire+0x60/0x300
<4> [242.608211] ? lock_release+0xd0/0x2b0
<4> [242.608215] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.608222] process_one_work+0x239/0x740
<4> [242.608229] worker_thread+0x200/0x3f0
<4> [242.608232] ? __pfx_worker_thread+0x10/0x10
<4> [242.608234] kthread+0x10d/0x150
<4> [242.608237] ? __pfx_kthread+0x10/0x10
<4> [242.608240] ret_from_fork+0x3bd/0x470
<4> [242.608242] ? __pfx_kthread+0x10/0x10
<4> [242.608245] ret_from_fork_asm+0x1a/0x30
<4> [242.608252] </TASK>
<4> [242.608253] irq event stamp: 626899
<4> [242.608255] hardirqs last enabled at (626905): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.608257] hardirqs last disabled at (626910): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.608259] softirqs last enabled at (626048): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.608262] softirqs last disabled at (626025): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.608264] ---[ end trace 0000000000000000 ]---
<7> [242.609293] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.609419] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [242.609951] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38039, lrc_seqno=38039, guc_id=0, flags=0x73 in no process [-1]
<7> [242.609984] xe 0000:03:00.0: [drm:drm_mode_addfb2] [FB:563]
<7> [242.609954] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.610031] ------------[ cut here ]------------
<4> [242.610032] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.610033] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [242.610119] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.610184]
<4> [242.610186] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.610189] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.610190] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.610191] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.610198] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.610280] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.610282] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.610284] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.610285] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.610286] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.610287] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.610289] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.610290] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [242.610292] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.610293] CR2: 00007fe38c095208 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.610295] PKRU: 55555554
<4> [242.610296] Call Trace:
<4> [242.610297] <TASK>
<4> [242.610301] ? lock_acquire+0x60/0x300
<4> [242.610306] ? lock_release+0xd0/0x2b0
<4> [242.610310] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.610317] process_one_work+0x239/0x740
<4> [242.610323] worker_thread+0x200/0x3f0
<4> [242.610326] ? __pfx_worker_thread+0x10/0x10
<4> [242.610329] kthread+0x10d/0x150
<4> [242.610331] ? __pfx_kthread+0x10/0x10
<4> [242.610334] ret_from_fork+0x3bd/0x470
<4> [242.610336] ? __pfx_kthread+0x10/0x10
<4> [242.610339] ret_from_fork_asm+0x1a/0x30
<4> [242.610347] </TASK>
<4> [242.610348] irq event stamp: 627749
<4> [242.610349] hardirqs last enabled at (627755): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.610351] hardirqs last disabled at (627760): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.610353] softirqs last enabled at (626048): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.610356] softirqs last disabled at (626025): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.610358] ---[ end trace 0000000000000000 ]---
<6> [242.610360] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.610442] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.610448] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.610498] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.610771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.610881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.610988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.611085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.611180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.611289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.611388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.611483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.611574] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.611701] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.611803] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.612951] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.623766] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.624099] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.625290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.625422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.625549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.625687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.625817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.625944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.626072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.626199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.626326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.626453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.626580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.626716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.626845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.626972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.627111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.627218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.627307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.627390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.627471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.627560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.627647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.627741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.627822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.627905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.627985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.628063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.628140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.628218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.628296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.628377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.628463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.628546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.628628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.628720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.628806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.628890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.628975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.629057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.629138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.629216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.629294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.629372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.629448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.629526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.629604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.629686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.629764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.629839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.629915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.629992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.630074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.630152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.630230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.630305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.630381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.630459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.630536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.630610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.630688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.630767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.630843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.630919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.631011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.631090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.631167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.631245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.631322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.631398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.631476] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.631618] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.631623] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38039, lrc_seqno=38039, guc_id=0, flags=0x73 in no process [-1]
<7> [242.631625] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.632550] ------------[ cut here ]------------
<4> [242.632551] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.632552] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [242.632632] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.632704]
<4> [242.632706] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.632710] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.632711] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.632712] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.632718] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.632795] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.632797] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [242.632800] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.632801] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.632802] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.632804] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.632805] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.632806] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [242.632808] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.632809] CR2: 0000718d92195000 CR3: 000000020918f000 CR4: 0000000000f52ef0
<4> [242.632811] PKRU: 55555554
<4> [242.632812] Call Trace:
<4> [242.632813] <TASK>
<4> [242.632817] ? lock_acquire+0x60/0x300
<4> [242.632822] ? lock_release+0xd0/0x2b0
<4> [242.632827] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.632833] process_one_work+0x239/0x740
<4> [242.632840] worker_thread+0x200/0x3f0
<4> [242.632843] ? __pfx_worker_thread+0x10/0x10
<4> [242.632846] kthread+0x10d/0x150
<4> [242.632848] ? __pfx_kthread+0x10/0x10
<4> [242.632851] ret_from_fork+0x3bd/0x470
<4> [242.632853] ? __pfx_kthread+0x10/0x10
<4> [242.632856] ret_from_fork_asm+0x1a/0x30
<4> [242.632863] </TASK>
<4> [242.632864] irq event stamp: 183605
<4> [242.632866] hardirqs last enabled at (183611): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.632868] hardirqs last disabled at (183616): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.632869] softirqs last enabled at (182588): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.632872] softirqs last disabled at (182583): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.632875] ---[ end trace 0000000000000000 ]---
<6> [242.632877] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.632952] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.632958] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.632974] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.633086] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.633238] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.633455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.633554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.633653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.633763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.633856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.633944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.634032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.634121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.634205] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.634305] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.634398] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.635432] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.645677] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.645935] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.646932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.647017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.647098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.647180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.647259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.647338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.647416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.647493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.647571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.647648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.647757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.647836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.647910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.647984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.648060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.648136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.648212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.648288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.648365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.648449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.648532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.648613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.648719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.648801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.648880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.648958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.649036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.649117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.649202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.649283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.649364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.649443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.649521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.649600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.649693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.649777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.649861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.649942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.650022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.650100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.650177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.650253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.650328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.650404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.650483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.650561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.650639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.650726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.650803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.650879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.650958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.651033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.651108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.651184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.651260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.651337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.651412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.651485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.651560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.651637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.651726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.651808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.651886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.651963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.652042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.652119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.652195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.652270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.652348] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.652637] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.652641] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38039, lrc_seqno=38039, guc_id=0, flags=0x73 in no process [-1]
<7> [242.652644] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.653566] ------------[ cut here ]------------
<4> [242.653568] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.653569] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:3/186
<4> [242.653660] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.653732]
<4> [242.653734] CPU: 11 UID: 0 PID: 186 Comm: kworker/u64:3 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.653737] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.653738] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.653740] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.653745] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.653821] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.653823] RSP: 0018:ffffc9000159fca0 EFLAGS: 00010246
<4> [242.653825] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.653826] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.653828] RBP: ffffc9000159fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.653829] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.653830] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.653831] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.653833] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.653834] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.653836] PKRU: 55555554
<4> [242.653837] Call Trace:
<4> [242.653838] <TASK>
<4> [242.653842] ? lock_acquire+0x60/0x300
<4> [242.653847] ? lock_release+0xd0/0x2b0
<4> [242.653852] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.653858] process_one_work+0x239/0x740
<4> [242.653865] worker_thread+0x200/0x3f0
<4> [242.653868] ? __pfx_worker_thread+0x10/0x10
<4> [242.653870] kthread+0x10d/0x150
<4> [242.653873] ? __pfx_kthread+0x10/0x10
<4> [242.653876] ret_from_fork+0x3bd/0x470
<4> [242.653878] ? __pfx_kthread+0x10/0x10
<4> [242.653881] ret_from_fork_asm+0x1a/0x30
<4> [242.653888] </TASK>
<4> [242.653889] irq event stamp: 481175
<4> [242.653891] hardirqs last enabled at (481181): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.653893] hardirqs last disabled at (481186): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.653895] softirqs last enabled at (479994): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.653897] softirqs last disabled at (479985): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.653900] ---[ end trace 0000000000000000 ]---
<7> [242.653944] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.654056] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [242.658253] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38042, lrc_seqno=38042, guc_id=0, flags=0x73 in no process [-1]
<7> [242.658258] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.658339] ------------[ cut here ]------------
<4> [242.658341] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.658342] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [242.658426] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.658490]
<4> [242.658493] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.658496] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.658497] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.658498] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.658504] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.658582] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.658583] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.658586] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.658587] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.658588] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.658590] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.658591] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.658592] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.658594] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.658595] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.658597] PKRU: 55555554
<4> [242.658598] Call Trace:
<4> [242.658599] <TASK>
<4> [242.658603] ? lock_acquire+0x60/0x300
<4> [242.658608] ? lock_release+0xd0/0x2b0
<4> [242.658613] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.658618] process_one_work+0x239/0x740
<4> [242.658625] worker_thread+0x200/0x3f0
<4> [242.658628] ? __pfx_worker_thread+0x10/0x10
<4> [242.658631] kthread+0x10d/0x150
<4> [242.658633] ? __pfx_kthread+0x10/0x10
<4> [242.658636] ret_from_fork+0x3bd/0x470
<4> [242.658639] ? __pfx_kthread+0x10/0x10
<4> [242.658641] ret_from_fork_asm+0x1a/0x30
<4> [242.658649] </TASK>
<4> [242.658650] irq event stamp: 629057
<4> [242.658651] hardirqs last enabled at (629063): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.658653] hardirqs last disabled at (629068): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.658655] softirqs last enabled at (626048): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.658658] softirqs last disabled at (626025): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.658660] ---[ end trace 0000000000000000 ]---
<6> [242.658662] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.658757] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.658764] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.659042] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.659254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.659351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.659449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.659540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.659630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.659736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.659827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.659918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.660003] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.660105] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.660198] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.661235] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.671684] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.671946] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.672935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.673021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.673101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.673183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.673263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.673342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.673419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.673494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.673570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.673646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.673734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.673811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.673887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.673960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.674035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.674110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.674182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.674255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.674331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.674416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.674500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.674581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.674660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.674755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.674835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.674914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.674991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.675067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.675143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.675222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.675302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.675422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.675559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.675687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.675818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.675949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.676088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.676223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.676326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.676413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.676496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.676577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.676656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.676753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.676838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.676921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.677001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.677080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.677158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.677236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.677318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.677396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.677474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.677552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.677629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.677718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.677795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.677871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.677947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.678027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.678104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.678182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.678259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.678335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.678413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.678489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.678564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.678641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.678730] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.678856] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.678860] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38042, lrc_seqno=38042, guc_id=0, flags=0x73 in no process [-1]
<7> [242.678863] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.678930] ------------[ cut here ]------------
<4> [242.678931] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.678933] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [242.679011] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.679075]
<4> [242.679077] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.679080] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.679082] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.679083] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.679089] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.679164] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.679166] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.679168] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.679170] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.679171] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.679172] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.679174] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.679175] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.679177] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.679178] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.679179] PKRU: 55555554
<4> [242.679181] Call Trace:
<4> [242.679182] <TASK>
<4> [242.679186] ? lock_acquire+0x60/0x300
<4> [242.679191] ? lock_release+0xd0/0x2b0
<4> [242.679196] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.679202] process_one_work+0x239/0x740
<4> [242.679208] worker_thread+0x200/0x3f0
<4> [242.679211] ? __pfx_worker_thread+0x10/0x10
<4> [242.679214] kthread+0x10d/0x150
<4> [242.679216] ? __pfx_kthread+0x10/0x10
<4> [242.679219] ret_from_fork+0x3bd/0x470
<4> [242.679221] ? __pfx_kthread+0x10/0x10
<4> [242.679224] ret_from_fork_asm+0x1a/0x30
<4> [242.679232] </TASK>
<4> [242.679233] irq event stamp: 632151
<4> [242.679234] hardirqs last enabled at (632157): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.679236] hardirqs last disabled at (632162): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.679238] softirqs last enabled at (630884): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.679241] softirqs last disabled at (630879): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.679243] ---[ end trace 0000000000000000 ]---
<6> [242.679245] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.679320] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.679326] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.680439] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.680548] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.680658] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.680897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.681000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.681103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.681197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.681285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.681372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.681458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.681546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.681630] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.681743] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.681837] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.682872] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.692674] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.692932] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.693886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.693971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.694051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.694131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.694210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.694288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.694365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.694442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.694518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.694595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.694675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.694752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.694828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.694905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.694981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.695057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.695133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.695208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.695286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.695373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.695458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.695540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.695619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.695713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.695796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.695874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.695952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.696028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.696105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.696185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.696268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.696347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.696426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.696508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.696593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.696682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.696775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.696860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.696945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.697023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.697101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.697178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.697255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.697332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.697411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.697488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.697566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.697642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.697728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.697805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.697884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.697961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.698038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.698115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.698190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.698268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.698344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.698419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.698494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.698572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.698649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.698740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.698820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.698897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.698974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.699051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.699127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.699203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.699281] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.699404] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.699408] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38042, lrc_seqno=38042, guc_id=0, flags=0x73 in no process [-1]
<7> [242.699411] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.700333] ------------[ cut here ]------------
<4> [242.700334] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.700336] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [242.700416] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.700482]
<4> [242.700485] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.700488] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.700489] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.700491] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.700496] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.700573] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.700574] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [242.700577] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.700578] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.700580] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.700581] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.700582] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.700583] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [242.700585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.700586] CR2: 00007fe39844b000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.700588] PKRU: 55555554
<4> [242.700589] Call Trace:
<4> [242.700590] <TASK>
<4> [242.700594] ? lock_acquire+0x60/0x300
<4> [242.700599] ? lock_release+0xd0/0x2b0
<4> [242.700604] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.700610] process_one_work+0x239/0x740
<4> [242.700616] worker_thread+0x200/0x3f0
<4> [242.700619] ? __pfx_worker_thread+0x10/0x10
<4> [242.700622] kthread+0x10d/0x150
<4> [242.700624] ? __pfx_kthread+0x10/0x10
<4> [242.700627] ret_from_fork+0x3bd/0x470
<4> [242.700630] ? __pfx_kthread+0x10/0x10
<4> [242.700632] ret_from_fork_asm+0x1a/0x30
<4> [242.700640] </TASK>
<4> [242.700641] irq event stamp: 187131
<4> [242.700642] hardirqs last enabled at (187137): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.700644] hardirqs last disabled at (187142): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.700646] softirqs last enabled at (186046): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.700649] softirqs last disabled at (186039): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.700651] ---[ end trace 0000000000000000 ]---
<7> [242.700696] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.700806] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.702158] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 0) -> ( 0 - 1989), size 0 -> 1989
<7> [242.702270] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:153:plane 1C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [242.702361] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:153:plane 1C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 1, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [242.702444] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:153:plane 1C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 16, 49, 145, 145, 273, 273, 273, 273, 30, 241, 255
<7> [242.702523] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:153:plane 1C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 19, 55, 161, 161, 302, 302, 0, 0, 31, 267, 267
<7> [242.702615] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [242.702748] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [242.702863] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [242.702971] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<5> [242.713964] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38043, lrc_seqno=38043, guc_id=0, flags=0x73 in no process [-1]
<7> [242.713967] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.714037] ------------[ cut here ]------------
<4> [242.714038] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.714039] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [242.714117] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.714181]
<4> [242.714183] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.714186] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.714187] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.714188] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.714194] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.714272] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.714274] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [242.714277] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.714280] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.714282] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.714284] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.714286] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.714288] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [242.714290] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.714292] CR2: 00007fe39844b000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.714295] PKRU: 55555554
<4> [242.714297] Call Trace:
<4> [242.714298] <TASK>
<4> [242.714304] ? lock_acquire+0x60/0x300
<4> [242.714312] ? lock_release+0xd0/0x2b0
<4> [242.714319] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.714330] process_one_work+0x239/0x740
<4> [242.714340] worker_thread+0x200/0x3f0
<4> [242.714345] ? __pfx_worker_thread+0x10/0x10
<4> [242.714349] kthread+0x10d/0x150
<4> [242.714353] ? __pfx_kthread+0x10/0x10
<4> [242.714357] ret_from_fork+0x3bd/0x470
<4> [242.714360] ? __pfx_kthread+0x10/0x10
<4> [242.714365] ret_from_fork_asm+0x1a/0x30
<4> [242.714378] </TASK>
<4> [242.714380] irq event stamp: 188355
<4> [242.714381] hardirqs last enabled at (188361): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.714384] hardirqs last disabled at (188366): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.714386] softirqs last enabled at (186046): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.714389] softirqs last disabled at (186039): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.714391] ---[ end trace 0000000000000000 ]---
<6> [242.714393] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.714475] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.714483] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.714645] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.714974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.715123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.715274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.715417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.715557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.715767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.715883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.715986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.716077] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.716181] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.716276] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.717335] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.727685] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.727948] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.728937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.729025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.729107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.729187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.729267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.729346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.729424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.729502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.729579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.729654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.729748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.729824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.729901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.729978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.730054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.730130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.730207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.730285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.730360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.730447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.730532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.730613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.730701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.730785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.730866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.730943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.731020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.731097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.731173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.731252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.731332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.731410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.731489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.731566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.731651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.731748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.731832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.731913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.731991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.732081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.732159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.732238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.732315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.732394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.732474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.732552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.732630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.732731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.732810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.732890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.732971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.733050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.733129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.733206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.733283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.733362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.733439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.733515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.733591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.733674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.733751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.733828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.733904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.733979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.734060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.734139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.734216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.734291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.734371] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.734494] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.734498] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38043, lrc_seqno=38043, guc_id=0, flags=0x73 in no process [-1]
<7> [242.734501] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.734569] ------------[ cut here ]------------
<4> [242.734570] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.734572] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:3/186
<4> [242.734650] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.734724]
<4> [242.734726] CPU: 10 UID: 0 PID: 186 Comm: kworker/u64:3 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.734729] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.734730] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.734732] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.734738] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.734813] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.734815] RSP: 0018:ffffc9000159fca0 EFLAGS: 00010246
<4> [242.734818] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.734819] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.734820] RBP: ffffc9000159fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.734821] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.734823] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.734824] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [242.734825] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.734827] CR2: 00007fe398449000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [242.734828] PKRU: 55555554
<4> [242.734829] Call Trace:
<4> [242.734831] <TASK>
<4> [242.734835] ? lock_acquire+0x60/0x300
<4> [242.734840] ? lock_release+0xd0/0x2b0
<4> [242.734845] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.734851] process_one_work+0x239/0x740
<4> [242.734857] worker_thread+0x200/0x3f0
<4> [242.734860] ? __pfx_worker_thread+0x10/0x10
<4> [242.734863] kthread+0x10d/0x150
<4> [242.734865] ? __pfx_kthread+0x10/0x10
<4> [242.734868] ret_from_fork+0x3bd/0x470
<4> [242.734870] ? __pfx_kthread+0x10/0x10
<4> [242.734873] ret_from_fork_asm+0x1a/0x30
<4> [242.734880] </TASK>
<4> [242.734881] irq event stamp: 485105
<4> [242.734882] hardirqs last enabled at (485111): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.734885] hardirqs last disabled at (485116): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.734886] softirqs last enabled at (483832): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.734889] softirqs last disabled at (483827): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.734892] ---[ end trace 0000000000000000 ]---
<6> [242.734894] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.734968] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.734976] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.736224] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.736334] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.736445] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.736658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.736775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.736870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.736960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.737047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.737136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.737224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.737313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.737396] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.737495] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.737587] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.738942] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.749685] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.749947] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.750973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.751061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.751144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.751226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.751306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.751386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.751465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.751543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.751621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.751715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.751794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.751871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.751948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.752025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.752103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.752180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.752257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.752333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.752410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.752498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.752584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.752673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.752754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.752838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.752920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.753000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.753077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.753155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.753230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.753309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.753389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.753466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.753544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.753623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.753717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.753801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.753885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.753967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.754047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.754126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.754204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.754281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.754356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.754435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.754514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.754592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.754675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.754753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.754829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.754907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.754986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.755063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.755140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.755216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.755292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.755371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.755447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.755523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.755599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.755683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.755762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.755839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.755917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.755992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.756068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.756145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.756221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.756297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.756375] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.756503] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.756718] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38043, lrc_seqno=38043, guc_id=0, flags=0x73 in no process [-1]
<7> [242.756722] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.757290] ------------[ cut here ]------------
<4> [242.757293] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.757295] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#0: kworker/u64:2/121
<4> [242.757437] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.757513]
<4> [242.757517] CPU: 0 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.757522] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.757524] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.757526] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.757536] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.757625] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.757627] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.757630] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.757631] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.757633] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.757634] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.757635] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.757636] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [242.757638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.757639] CR2: 00007fe39844c000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.757641] PKRU: 55555554
<4> [242.757642] Call Trace:
<4> [242.757643] <TASK>
<4> [242.757647] ? lock_acquire+0x60/0x300
<4> [242.757655] ? lock_release+0xd0/0x2b0
<4> [242.757660] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.757677] process_one_work+0x239/0x740
<4> [242.757685] worker_thread+0x200/0x3f0
<4> [242.757688] ? __pfx_worker_thread+0x10/0x10
<4> [242.757691] kthread+0x10d/0x150
<4> [242.757694] ? __pfx_kthread+0x10/0x10
<4> [242.757697] ret_from_fork+0x3bd/0x470
<4> [242.757699] ? __pfx_kthread+0x10/0x10
<4> [242.757702] ret_from_fork_asm+0x1a/0x30
<4> [242.757710] </TASK>
<4> [242.757711] irq event stamp: 635591
<4> [242.757712] hardirqs last enabled at (635597): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.757714] hardirqs last disabled at (635602): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.757716] softirqs last enabled at (635532): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.757719] softirqs last disabled at (635527): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.757722] ---[ end trace 0000000000000000 ]---
<7> [242.757787] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.757898] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [242.764898] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38051, lrc_seqno=38051, guc_id=0, flags=0x73 in no process [-1]
<7> [242.764902] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.764977] ------------[ cut here ]------------
<4> [242.764979] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.764980] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#0: kworker/u64:2/121
<4> [242.765059] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.765123]
<4> [242.765125] CPU: 0 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.765128] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.765129] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.765130] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.765135] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.765212] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.765214] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.765216] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.765217] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.765219] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.765220] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.765221] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.765222] FS: 0000000000000000(0000) GS:ffff8888dac83000(0000) knlGS:0000000000000000
<4> [242.765224] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.765225] CR2: 00007fe39844c000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.765227] PKRU: 55555554
<4> [242.765228] Call Trace:
<4> [242.765229] <TASK>
<4> [242.765232] ? lock_acquire+0x60/0x300
<4> [242.765237] ? lock_release+0xd0/0x2b0
<4> [242.765242] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.765248] process_one_work+0x239/0x740
<4> [242.765254] worker_thread+0x200/0x3f0
<4> [242.765257] ? __pfx_worker_thread+0x10/0x10
<4> [242.765260] kthread+0x10d/0x150
<4> [242.765262] ? __pfx_kthread+0x10/0x10
<4> [242.765265] ret_from_fork+0x3bd/0x470
<4> [242.765267] ? __pfx_kthread+0x10/0x10
<4> [242.765270] ret_from_fork_asm+0x1a/0x30
<4> [242.765277] </TASK>
<4> [242.765278] irq event stamp: 636677
<4> [242.765280] hardirqs last enabled at (636683): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.765282] hardirqs last disabled at (636688): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.765283] softirqs last enabled at (635532): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.765286] softirqs last disabled at (635527): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.765289] ---[ end trace 0000000000000000 ]---
<6> [242.765290] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.765367] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.765373] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.765652] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.765879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.765976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.766073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.766167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.766256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.766343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.766430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.766523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.766608] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.766725] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.766820] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.767871] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.777675] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.777935] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.778952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.779041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.779122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.779202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.779281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.779357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.779434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.779510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.779586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.779661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.779751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.779828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.779903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.779976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.780052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.780126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.780200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.780276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.780351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.780437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.780521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.780606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.780695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.780777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.780857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.780935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.781012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.781089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.781170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.781250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.781331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.781410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.781489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.781569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.781653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.781757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.781845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.781928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.782008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.782085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.782163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.782241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.782318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.782396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.782474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.782551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.782631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.782723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.782802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.782881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.782961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.783037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.783115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.783192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.783266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.783344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.783421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.783496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.783571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.783649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.783741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.783819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.783896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.783974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.784054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.784132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.784208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.784283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.784363] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.784488] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.784492] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38051, lrc_seqno=38051, guc_id=0, flags=0x73 in no process [-1]
<7> [242.784495] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.784714] ------------[ cut here ]------------
<4> [242.784719] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.784723] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:2/121
<4> [242.785111] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.785268]
<4> [242.785275] CPU: 1 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.785284] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.785288] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.785291] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.785313] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.785466] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.785468] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.785471] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.785472] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.785474] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.785475] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.785476] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.785478] FS: 0000000000000000(0000) GS:ffff8888dad03000(0000) knlGS:0000000000000000
<4> [242.785479] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.785481] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.785482] PKRU: 55555554
<4> [242.785484] Call Trace:
<4> [242.785485] <TASK>
<4> [242.785489] ? lock_acquire+0x60/0x300
<4> [242.785498] ? lock_release+0xd0/0x2b0
<4> [242.785506] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.785517] process_one_work+0x239/0x740
<4> [242.785528] worker_thread+0x200/0x3f0
<4> [242.785533] ? __pfx_worker_thread+0x10/0x10
<4> [242.785537] kthread+0x10d/0x150
<4> [242.785540] ? __pfx_kthread+0x10/0x10
<4> [242.785545] ret_from_fork+0x3bd/0x470
<4> [242.785548] ? __pfx_kthread+0x10/0x10
<4> [242.785552] ret_from_fork_asm+0x1a/0x30
<4> [242.785565] </TASK>
<4> [242.785567] irq event stamp: 639847
<4> [242.785569] hardirqs last enabled at (639853): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.785573] hardirqs last disabled at (639858): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.785576] softirqs last enabled at (638940): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.785579] softirqs last disabled at (638935): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.785583] ---[ end trace 0000000000000000 ]---
<7> [242.785513] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<6> [242.785586] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.785727] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.785737] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.785679] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.785909] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.786125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.786221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.786317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.786409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.786496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.786583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.786678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.786765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.786848] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.786948] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.787042] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.788076] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.798672] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.798931] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.799940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.800026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.800105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.800186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.800264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.800342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.800419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.800495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.800574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.800649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.800744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.800823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.800899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.800976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.801053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.801132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.801210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.801286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.801361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.801447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.801531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.801611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.801699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.801778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.801858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.801936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.802012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.802089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.802166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.802246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.802324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.802403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.802481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.802560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.802645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.802741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.802826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.802907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.802990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.803067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.803145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.803222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.803298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.803376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.803455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.803532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.803608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.803691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.803767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.803845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.803925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.804002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.804079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.804156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.804232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.804309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.804386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.804461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.804537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.804616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.804704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.804782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.804859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.804936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.805013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.805090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.805165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.805240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.805317] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.805441] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.805445] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38051, lrc_seqno=38051, guc_id=0, flags=0x73 in no process [-1]
<7> [242.805447] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.805514] ------------[ cut here ]------------
<4> [242.805515] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.805516] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#1: kworker/u64:2/121
<4> [242.805603] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.805967]
<4> [242.805969] CPU: 1 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.805973] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.805974] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.805976] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.805982] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.806067] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.806068] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.806070] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.806072] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.806073] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.806074] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.806076] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.806077] FS: 0000000000000000(0000) GS:ffff8888dad03000(0000) knlGS:0000000000000000
<4> [242.806078] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.806080] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.806081] PKRU: 55555554
<4> [242.806083] Call Trace:
<4> [242.806084] <TASK>
<4> [242.806088] ? lock_acquire+0x60/0x300
<4> [242.806093] ? lock_release+0xd0/0x2b0
<4> [242.806098] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.806104] process_one_work+0x239/0x740
<4> [242.806110] worker_thread+0x200/0x3f0
<4> [242.806113] ? __pfx_worker_thread+0x10/0x10
<4> [242.806116] kthread+0x10d/0x150
<4> [242.806118] ? __pfx_kthread+0x10/0x10
<4> [242.806121] ret_from_fork+0x3bd/0x470
<4> [242.806123] ? __pfx_kthread+0x10/0x10
<4> [242.806126] ret_from_fork_asm+0x1a/0x30
<4> [242.806134] </TASK>
<4> [242.806135] irq event stamp: 642995
<4> [242.806136] hardirqs last enabled at (643001): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.806138] hardirqs last disabled at (643006): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.806140] softirqs last enabled at (641820): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.806142] softirqs last disabled at (641813): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.806145] ---[ end trace 0000000000000000 ]---
<7> [242.807403] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.807520] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.813948] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [242.898270] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [242.898887] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [242.899351] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [242.899828] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [242.900241] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [242.900639] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [242.901091] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [242.901474] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [242.901898] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [242.902282] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [242.902756] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [242.902930] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [242.903002] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [242.903072] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [242.903138] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [242.903204] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [242.903269] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [242.903334] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [242.903399] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [242.903463] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [242.903529] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [242.903601] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [242.903720] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [242.903834] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [242.903937] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [242.948207] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38052, lrc_seqno=38052, guc_id=0, flags=0x73 in no process [-1]
<7> [242.948232] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.948622] ------------[ cut here ]------------
<4> [242.948629] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.948637] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [242.949160] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.949606]
<4> [242.949620] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.949637] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.949644] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.949653] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.949700] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.950112] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.950122] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [242.950136] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.950144] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.950152] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.950158] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.950165] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.950174] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [242.950182] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.950191] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.950200] PKRU: 55555554
<4> [242.950207] Call Trace:
<4> [242.950214] <TASK>
<4> [242.950236] ? lock_acquire+0x60/0x300
<4> [242.950264] ? lock_release+0xd0/0x2b0
<4> [242.950290] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.950321] process_one_work+0x239/0x740
<4> [242.950357] worker_thread+0x200/0x3f0
<4> [242.950374] ? __pfx_worker_thread+0x10/0x10
<4> [242.950388] kthread+0x10d/0x150
<4> [242.950401] ? __pfx_kthread+0x10/0x10
<4> [242.950417] ret_from_fork+0x3bd/0x470
<4> [242.950429] ? __pfx_kthread+0x10/0x10
<4> [242.950444] ret_from_fork_asm+0x1a/0x30
<4> [242.950484] </TASK>
<4> [242.950491] irq event stamp: 644437
<4> [242.950498] hardirqs last enabled at (644443): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.950511] hardirqs last disabled at (644448): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.950520] softirqs last enabled at (643690): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.950535] softirqs last disabled at (643685): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.950550] ---[ end trace 0000000000000000 ]---
<6> [242.950728] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.951330] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.951391] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.951556] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.952375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.952986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.953626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.954338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.954884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.955347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.955872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.956352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.956855] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.957392] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.957950] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.959496] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.969675] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.969995] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.971100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.971203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.971306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.971413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.971518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.971619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.971737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.971860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.971965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.972069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.972174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.972275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.972375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.972475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.972576] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.972685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.972792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.972887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.972979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.973083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.973185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.973281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.973377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.973473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.973565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.973650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.973762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.973857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.973937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.974020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.974101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.974179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.974258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.974335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.974418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.974506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.974589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.974683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.974773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.974845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.974920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.974991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.975060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.975131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.975203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.975274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.975344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.975414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.975483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.975554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.975626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.975712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.975804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.975882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.975957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.976034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.976111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.976187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.976262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.976341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.976417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.976493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.976569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.976644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.976738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.976827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.976903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.976978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.977057] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.977195] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.977199] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38052, lrc_seqno=38052, guc_id=0, flags=0x73 in no process [-1]
<7> [242.977202] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.977270] ------------[ cut here ]------------
<4> [242.977272] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.977273] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:4/203
<4> [242.977353] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.977419]
<4> [242.977421] CPU: 2 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.977425] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.977426] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.977427] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.977433] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.977516] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.977518] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [242.977520] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.977521] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.977523] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.977524] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.977525] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.977527] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [242.977528] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.977530] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [242.977531] PKRU: 55555554
<4> [242.977532] Call Trace:
<4> [242.977533] <TASK>
<4> [242.977538] ? lock_acquire+0x60/0x300
<4> [242.977543] ? lock_release+0xd0/0x2b0
<4> [242.977548] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.977554] process_one_work+0x239/0x740
<4> [242.977561] worker_thread+0x200/0x3f0
<4> [242.977564] ? __pfx_worker_thread+0x10/0x10
<4> [242.977567] kthread+0x10d/0x150
<4> [242.977569] ? __pfx_kthread+0x10/0x10
<4> [242.977572] ret_from_fork+0x3bd/0x470
<4> [242.977574] ? __pfx_kthread+0x10/0x10
<4> [242.977577] ret_from_fork_asm+0x1a/0x30
<4> [242.977584] </TASK>
<4> [242.977585] irq event stamp: 192991
<4> [242.977587] hardirqs last enabled at (192997): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.977589] hardirqs last disabled at (193002): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.977590] softirqs last enabled at (191930): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.977593] softirqs last disabled at (191885): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.977596] ---[ end trace 0000000000000000 ]---
<6> [242.977887] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [242.978253] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [242.978275] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [242.978721] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [242.978835] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [242.978947] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [242.979163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [242.979258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [242.979355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [242.979446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [242.979535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [242.979622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [242.979769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [242.979865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [242.979952] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [242.980053] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [242.980146] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [242.981194] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [242.991668] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [242.991944] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [242.993014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [242.993104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [242.993187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [242.993268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [242.993347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [242.993424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [242.993501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [242.993576] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [242.993653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [242.993750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [242.993837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [242.993914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [242.993990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [242.994065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [242.994140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [242.994215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [242.994291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [242.994367] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [242.994443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [242.994530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [242.994615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [242.994714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [242.994807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [242.994887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [242.994968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [242.995049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [242.995127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [242.995204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [242.995281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [242.995362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [242.995444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [242.995523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [242.995602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [242.995691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [242.995787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [242.995873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [242.995956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [242.996038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [242.996118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [242.996195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [242.996277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [242.996355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [242.996430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [242.996509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [242.996588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [242.996675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [242.996765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [242.996844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [242.996923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [242.997005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [242.997090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [242.997167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [242.997246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [242.997323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [242.997398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [242.997476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [242.997551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [242.997630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [242.997722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [242.997815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [242.997894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [242.997992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [242.998071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [242.998147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [242.998225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [242.998303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [242.998377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [242.998454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [242.998536] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [242.998667] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [242.998674] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38052, lrc_seqno=38052, guc_id=0, flags=0x73 in no process [-1]
<7> [242.998678] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [242.998758] ------------[ cut here ]------------
<4> [242.998759] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [242.998760] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:4/203
<4> [242.998840] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [242.998904]
<4> [242.998906] CPU: 3 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [242.998910] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [242.998911] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [242.998912] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [242.998918] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [242.998995] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [242.998996] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [242.998999] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [242.999000] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [242.999001] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [242.999003] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [242.999004] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [242.999005] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [242.999007] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [242.999008] CR2: 00007fe39844d000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [242.999010] PKRU: 55555554
<4> [242.999011] Call Trace:
<4> [242.999012] <TASK>
<4> [242.999016] ? lock_acquire+0x60/0x300
<4> [242.999021] ? lock_release+0xd0/0x2b0
<4> [242.999026] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [242.999032] process_one_work+0x239/0x740
<4> [242.999039] worker_thread+0x200/0x3f0
<4> [242.999042] ? __pfx_worker_thread+0x10/0x10
<4> [242.999044] kthread+0x10d/0x150
<4> [242.999047] ? __pfx_kthread+0x10/0x10
<4> [242.999049] ret_from_fork+0x3bd/0x470
<4> [242.999051] ? __pfx_kthread+0x10/0x10
<4> [242.999054] ret_from_fork_asm+0x1a/0x30
<4> [242.999062] </TASK>
<4> [242.999063] irq event stamp: 196161
<4> [242.999064] hardirqs last enabled at (196167): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [242.999066] hardirqs last disabled at (196172): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [242.999068] softirqs last enabled at (195244): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.999071] softirqs last disabled at (195233): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [242.999073] ---[ end trace 0000000000000000 ]---
<7> [243.000385] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.000500] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [243.006842] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38060, lrc_seqno=38060, guc_id=0, flags=0x73 in no process [-1]
<7> [243.006845] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.006915] ------------[ cut here ]------------
<4> [243.006916] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.006918] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:4/203
<4> [243.006995] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.007060]
<4> [243.007062] CPU: 3 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.007065] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.007066] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.007068] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.007073] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.007150] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.007152] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [243.007154] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.007155] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.007156] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.007158] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.007159] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.007160] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [243.007162] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.007163] CR2: 00007fe39844d000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.007164] PKRU: 55555554
<4> [243.007165] Call Trace:
<4> [243.007166] <TASK>
<4> [243.007170] ? lock_acquire+0x60/0x300
<4> [243.007175] ? lock_release+0xd0/0x2b0
<4> [243.007180] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.007186] process_one_work+0x239/0x740
<4> [243.007192] worker_thread+0x200/0x3f0
<4> [243.007195] ? __pfx_worker_thread+0x10/0x10
<4> [243.007198] kthread+0x10d/0x150
<4> [243.007200] ? __pfx_kthread+0x10/0x10
<4> [243.007203] ret_from_fork+0x3bd/0x470
<4> [243.007205] ? __pfx_kthread+0x10/0x10
<4> [243.007208] ret_from_fork_asm+0x1a/0x30
<4> [243.007216] </TASK>
<4> [243.007217] irq event stamp: 197131
<4> [243.007218] hardirqs last enabled at (197137): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.007220] hardirqs last disabled at (197142): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.007222] softirqs last enabled at (195244): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.007225] softirqs last disabled at (195233): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.007227] ---[ end trace 0000000000000000 ]---
<6> [243.007229] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.007305] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.007311] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.007727] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.007974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.008076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.008176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.008271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.008359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.008447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.008535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.008622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.008723] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.008834] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.008929] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.009969] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.020682] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.020941] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.021915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.022007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.022089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.022170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.022249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.022325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.022403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.022477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.022553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.022632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.022728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.022819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.022897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.022974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.023050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.023127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.023201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.023276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.023352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.023439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.023522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.023604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.023694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.023791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.023875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.023953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.024031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.024106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.024182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.024261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.024344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.024424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.024500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.024579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.024677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.024771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.024856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.024938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.025019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.025096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.025175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.025254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.025329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.025409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.025489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.025568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.025645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.025738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.025828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.025907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.025988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.026068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.026146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.026224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.026299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.026376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.026453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.026530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.026606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.026695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.026783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.026864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.026945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.027022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.027099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.027175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.027251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.027325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.027403] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.027546] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.027550] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38060, lrc_seqno=38060, guc_id=0, flags=0x73 in no process [-1]
<7> [243.027553] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.027621] ------------[ cut here ]------------
<4> [243.027622] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.027623] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:3/186
<4> [243.027714] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.027800]
<4> [243.027802] CPU: 11 UID: 0 PID: 186 Comm: kworker/u64:3 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.027805] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.027807] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.027808] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.027814] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.027891] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.027892] RSP: 0018:ffffc9000159fca0 EFLAGS: 00010246
<4> [243.027895] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.027896] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.027897] RBP: ffffc9000159fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.027899] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.027900] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.027901] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [243.027903] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.027904] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.027906] PKRU: 55555554
<4> [243.027907] Call Trace:
<4> [243.027908] <TASK>
<4> [243.027912] ? lock_acquire+0x60/0x300
<4> [243.027917] ? lock_release+0xd0/0x2b0
<4> [243.027922] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.027928] process_one_work+0x239/0x740
<4> [243.027935] worker_thread+0x200/0x3f0
<4> [243.027938] ? __pfx_worker_thread+0x10/0x10
<4> [243.027941] kthread+0x10d/0x150
<4> [243.027943] ? __pfx_kthread+0x10/0x10
<4> [243.027946] ret_from_fork+0x3bd/0x470
<4> [243.027948] ? __pfx_kthread+0x10/0x10
<4> [243.027951] ret_from_fork_asm+0x1a/0x30
<4> [243.027959] </TASK>
<4> [243.027960] irq event stamp: 489429
<4> [243.027961] hardirqs last enabled at (489435): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.027963] hardirqs last disabled at (489440): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.027965] softirqs last enabled at (488338): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.027967] softirqs last disabled at (488333): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.027970] ---[ end trace 0000000000000000 ]---
<6> [243.028671] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.028831] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.028845] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.029221] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.029336] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.029449] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.029692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.029796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.029897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.029996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.030087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.030175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.030263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.030350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.030434] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.030536] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.030631] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.031678] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.041666] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.041926] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.042883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.042970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.043053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.043136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.043220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.043304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.043382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.043459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.043536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.043612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.043702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.043790] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.043868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.043944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.044019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.044093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.044171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.044245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.044318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.044403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.044488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.044567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.044647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.044745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.044843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.044922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.044999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.045077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.045155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.045236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.045318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.045400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.045481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.045564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.045649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.045754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.045848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.045931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.046013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.046091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.046176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.046258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.046335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.046413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.046493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.046571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.046654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.046750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.046839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.046918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.046998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.047075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.047153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.047229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.047305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.047382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.047458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.047534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.047608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.047705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.047794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.047870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.047946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.048021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.048097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.048173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.048249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.048324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.048403] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.048528] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.048532] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38060, lrc_seqno=38060, guc_id=0, flags=0x73 in no process [-1]
<7> [243.048535] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.048798] ------------[ cut here ]------------
<4> [243.048801] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.048803] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:2/121
<4> [243.048921] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.048994]
<4> [243.048998] CPU: 4 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.049002] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.049003] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.049005] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.049013] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.049094] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.049096] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [243.049098] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.049100] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.049101] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.049103] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.049104] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.049105] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [243.049107] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.049108] CR2: 00007fe39844b000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [243.049110] PKRU: 55555554
<4> [243.049111] Call Trace:
<4> [243.049112] <TASK>
<4> [243.049116] ? lock_acquire+0x60/0x300
<4> [243.049123] ? lock_release+0xd0/0x2b0
<4> [243.049128] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.049135] process_one_work+0x239/0x740
<4> [243.049142] worker_thread+0x200/0x3f0
<4> [243.049145] ? __pfx_worker_thread+0x10/0x10
<4> [243.049148] kthread+0x10d/0x150
<4> [243.049151] ? __pfx_kthread+0x10/0x10
<4> [243.049154] ret_from_fork+0x3bd/0x470
<4> [243.049156] ? __pfx_kthread+0x10/0x10
<4> [243.049159] ret_from_fork_asm+0x1a/0x30
<4> [243.049167] </TASK>
<4> [243.049168] irq event stamp: 648061
<4> [243.049170] hardirqs last enabled at (648067): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.049172] hardirqs last disabled at (648072): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.049174] softirqs last enabled at (647062): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.049177] softirqs last disabled at (647055): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.049180] ---[ end trace 0000000000000000 ]---
<7> [243.049553] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.049670] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.051089] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [243.051198] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [243.051292] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.051378] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051460] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051541] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051623] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.051719] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051797] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051872] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.051962] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [243.052162] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [243.052243] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [243.052320] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.052396] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052473] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052547] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052621] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.052704] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052778] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052852] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.052929] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [243.053045] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [243.053159] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [243.053268] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [243.064063] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [243.147773] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.148329] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.148847] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.149275] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.149711] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.150115] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.150514] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.150944] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.151341] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.151747] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.152194] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.152618] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.152711] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.152790] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.152864] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.152937] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.153007] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.153076] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.153144] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.153210] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.153276] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.153346] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.153447] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [243.153546] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [243.153643] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [243.198541] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38061, lrc_seqno=38061, guc_id=0, flags=0x73 in no process [-1]
<7> [243.198567] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.199073] ------------[ cut here ]------------
<4> [243.199080] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.199088] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [243.199526] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.199949]
<4> [243.199963] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.199980] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.199987] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.199995] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.200025] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.200429] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.200439] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [243.200453] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.200461] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.200469] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.200476] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.200483] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.200490] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [243.200499] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.200508] CR2: 00007fe39844e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.200517] PKRU: 55555554
<4> [243.200523] Call Trace:
<4> [243.200531] <TASK>
<4> [243.200553] ? lock_acquire+0x60/0x300
<4> [243.200581] ? lock_release+0xd0/0x2b0
<4> [243.200607] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.200638] process_one_work+0x239/0x740
<4> [243.200700] worker_thread+0x200/0x3f0
<4> [243.200718] ? __pfx_worker_thread+0x10/0x10
<4> [243.200733] kthread+0x10d/0x150
<4> [243.200749] ? __pfx_kthread+0x10/0x10
<4> [243.200774] ret_from_fork+0x3bd/0x470
<4> [243.200792] ? __pfx_kthread+0x10/0x10
<4> [243.200811] ret_from_fork_asm+0x1a/0x30
<4> [243.200850] </TASK>
<4> [243.200856] irq event stamp: 1309483
<4> [243.200862] hardirqs last enabled at (1309489): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.200875] hardirqs last disabled at (1309494): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.200884] softirqs last enabled at (1307324): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.200899] softirqs last disabled at (1307319): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.200912] ---[ end trace 0000000000000000 ]---
<6> [243.200923] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.202526] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.202564] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.202928] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.203638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.204255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.204793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.205263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.205742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.206189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.206642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.207178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.207637] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.208231] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.208745] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.210563] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.221934] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.222227] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.223513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.223634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.223757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.223866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.223977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.224084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.224187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.224288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.224387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.224482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.224577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.224674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.224769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.224859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.224948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.225039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.225129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.225220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.225305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.225404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.225501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.225590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.225685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.225771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.225855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.225938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.226020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.226101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.226183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.226268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.226352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.226435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.226520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.226608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.226705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.226795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.226886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.226971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.227056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.227138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.227221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.227303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.227383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.227466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.227549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.227631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.227720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.227804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.227884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.227965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.228050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.228130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.228211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.228293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.228372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.228455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.228537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.228616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.228703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.228787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.228868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.228948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.229028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.229109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.229192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.229273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.229353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.229433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.229516] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.229646] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.230499] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38061, lrc_seqno=38061, guc_id=0, flags=0x73 in no process [-1]
<7> [243.230502] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.230579] ------------[ cut here ]------------
<4> [243.230580] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.230581] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:8/983
<4> [243.230668] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.230739]
<4> [243.230741] CPU: 8 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.230745] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.230746] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.230748] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.230754] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.230834] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.230836] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [243.230838] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.230840] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.230841] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.230842] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.230843] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.230845] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [243.230846] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.230848] CR2: 0000718d911e3000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [243.230849] PKRU: 55555554
<4> [243.230851] Call Trace:
<4> [243.230852] <TASK>
<4> [243.230856] ? lock_acquire+0x60/0x300
<4> [243.230862] ? lock_release+0xd0/0x2b0
<4> [243.230867] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.230873] process_one_work+0x239/0x740
<4> [243.230881] worker_thread+0x200/0x3f0
<4> [243.230884] ? __pfx_worker_thread+0x10/0x10
<4> [243.230886] kthread+0x10d/0x150
<4> [243.230889] ? __pfx_kthread+0x10/0x10
<4> [243.230892] ret_from_fork+0x3bd/0x470
<4> [243.230895] ? __pfx_kthread+0x10/0x10
<4> [243.230897] ret_from_fork_asm+0x1a/0x30
<4> [243.230905] </TASK>
<4> [243.230907] irq event stamp: 1312639
<4> [243.230908] hardirqs last enabled at (1312645): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.230910] hardirqs last disabled at (1312650): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.230912] softirqs last enabled at (1311460): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.230915] softirqs last disabled at (1311455): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.230918] ---[ end trace 0000000000000000 ]---
<6> [243.230920] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.230999] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.231005] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.231419] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.231535] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.231906] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.232135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.232242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.232346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.232443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.232541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.232635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.232742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.232838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.232928] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.233036] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.233138] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.234179] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.244659] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.244922] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.245925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.246019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.246107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.246194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.246279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.246363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.246444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.246525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.246606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.246696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.246777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.246858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.246937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.247018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.247098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.247179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.247259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.247338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.247421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.247514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.247605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.247706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.247795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.247881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.247968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.248052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.248133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.248217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.248298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.248381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.248465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.248549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.248633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.248753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.248845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.248935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.249023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.249108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.249194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.249276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.249362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.249444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.249525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.249608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.249701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.249785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.249868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.249950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.250032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.250115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.250220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.250309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.250395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.250477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.250560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.250643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.250738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.250818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.250899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.250984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.251067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.251153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.251236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.251318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.251402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.251484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.251716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.251803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.251887] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.252015] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.252019] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38061, lrc_seqno=38061, guc_id=0, flags=0x73 in no process [-1]
<7> [243.252022] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.252093] ------------[ cut here ]------------
<4> [243.252095] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.252096] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:8/983
<4> [243.252178] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.252247]
<4> [243.252249] CPU: 8 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.252252] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.252254] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.252255] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.252261] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.252342] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.252343] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [243.252346] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.252348] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.252349] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.252350] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.252351] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.252353] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [243.252354] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.252356] CR2: 0000718d911e3000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [243.252357] PKRU: 55555554
<4> [243.252358] Call Trace:
<4> [243.252359] <TASK>
<4> [243.252363] ? lock_acquire+0x60/0x300
<4> [243.252369] ? lock_release+0xd0/0x2b0
<4> [243.252374] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.252380] process_one_work+0x239/0x740
<4> [243.252388] worker_thread+0x200/0x3f0
<4> [243.252391] ? __pfx_worker_thread+0x10/0x10
<4> [243.252393] kthread+0x10d/0x150
<4> [243.252396] ? __pfx_kthread+0x10/0x10
<4> [243.252399] ret_from_fork+0x3bd/0x470
<4> [243.252402] ? __pfx_kthread+0x10/0x10
<4> [243.252404] ret_from_fork_asm+0x1a/0x30
<4> [243.252412] </TASK>
<4> [243.252413] irq event stamp: 1315815
<4> [243.252415] hardirqs last enabled at (1315821): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.252417] hardirqs last disabled at (1315826): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.252419] softirqs last enabled at (1314950): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.252422] softirqs last disabled at (1314939): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.252425] ---[ end trace 0000000000000000 ]---
<7> [243.253738] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.253863] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [243.259825] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38069, lrc_seqno=38069, guc_id=0, flags=0x73 in no process [-1]
<7> [243.259830] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.259908] ------------[ cut here ]------------
<4> [243.259909] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.259911] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [243.259997] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.260066]
<4> [243.260068] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.260071] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.260073] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.260074] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.260080] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.260160] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.260162] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [243.260165] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.260166] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.260168] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.260169] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.260170] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.260172] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [243.260173] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.260174] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.260176] PKRU: 55555554
<4> [243.260177] Call Trace:
<4> [243.260178] <TASK>
<4> [243.260182] ? lock_acquire+0x60/0x300
<4> [243.260188] ? lock_release+0xd0/0x2b0
<4> [243.260193] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.260199] process_one_work+0x239/0x740
<4> [243.260207] worker_thread+0x200/0x3f0
<4> [243.260210] ? __pfx_worker_thread+0x10/0x10
<4> [243.260213] kthread+0x10d/0x150
<4> [243.260215] ? __pfx_kthread+0x10/0x10
<4> [243.260218] ret_from_fork+0x3bd/0x470
<4> [243.260221] ? __pfx_kthread+0x10/0x10
<4> [243.260223] ret_from_fork_asm+0x1a/0x30
<4> [243.260231] </TASK>
<4> [243.260232] irq event stamp: 651173
<4> [243.260233] hardirqs last enabled at (651179): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.260236] hardirqs last disabled at (651184): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.260237] softirqs last enabled at (648738): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.260240] softirqs last disabled at (648729): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.260243] ---[ end trace 0000000000000000 ]---
<6> [243.260245] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.260324] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.260330] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.260779] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.260999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.261101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.261202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.261298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.261392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.261484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.261577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.261676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.261766] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.261872] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.261974] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.263028] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.273658] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.273921] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.274861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.274953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.275039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.275124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.275208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.275292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.275375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.275457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.275542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.275624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.275715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.275796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.275876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.275956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.276036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.276115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.276194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.276274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.276354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.276445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.276535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.276622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.276715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.276802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.276887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.276970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.277053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.277136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.277219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.277303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.277387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.277469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.277555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.277639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.277738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.277828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.277918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.278005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.278092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.278175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.278261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.278344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.278427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.278511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.278599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.278688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.278773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.278856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.278936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.279019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.279104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.279185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.279270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.279351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.279431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.279512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.279593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.279678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.279763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.279849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.279931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.280018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.280101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.280183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.280266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.280348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.280428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.280509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.280593] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.280730] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.280734] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38069, lrc_seqno=38069, guc_id=0, flags=0x73 in no process [-1]
<7> [243.280737] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.280808] ------------[ cut here ]------------
<4> [243.280809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.280811] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:8/983
<4> [243.280894] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.280958]
<4> [243.280961] CPU: 8 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.280964] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.280965] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.280966] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.280972] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.281052] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.281053] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [243.281056] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.281057] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.281058] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.281060] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.281061] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.281062] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [243.281064] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.281065] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.281067] PKRU: 55555554
<4> [243.281068] Call Trace:
<4> [243.281069] <TASK>
<4> [243.281073] ? lock_acquire+0x60/0x300
<4> [243.281078] ? lock_release+0xd0/0x2b0
<4> [243.281083] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.281089] process_one_work+0x239/0x740
<4> [243.281096] worker_thread+0x200/0x3f0
<4> [243.281099] ? __pfx_worker_thread+0x10/0x10
<4> [243.281102] kthread+0x10d/0x150
<4> [243.281104] ? __pfx_kthread+0x10/0x10
<4> [243.281107] ret_from_fork+0x3bd/0x470
<4> [243.281109] ? __pfx_kthread+0x10/0x10
<4> [243.281112] ret_from_fork_asm+0x1a/0x30
<4> [243.281120] </TASK>
<4> [243.281121] irq event stamp: 1319129
<4> [243.281122] hardirqs last enabled at (1319135): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.281124] hardirqs last disabled at (1319140): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.281126] softirqs last enabled at (1318286): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.281129] softirqs last disabled at (1318275): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.281132] ---[ end trace 0000000000000000 ]---
<6> [243.281133] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.281213] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.281219] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.282451] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.282571] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.282692] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.282915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.283016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.283117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.283209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.283299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.283386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.283473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.283562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.283645] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.283760] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.283855] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.284891] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.295620] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.295889] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.296887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.296971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.297051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.297132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.297211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.297290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.297370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.297450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.297529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.297608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.297694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.297770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.297847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.297923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.298000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.298076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.298150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.298226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.298301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.298386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.298472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.298553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.298632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.298723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.298804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.298882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.298958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.299036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.299113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.299195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.299275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.299352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.299431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.299509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.299595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.299685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.299769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.299849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.299928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.300005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.300081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.300156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.300232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.300309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.300386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.300464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.300539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.300615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.300697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.300773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.300852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.300929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.301006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.301082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.301157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.301235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.301310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.301386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.301461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.301538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.301614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.301697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.301774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.301850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.301927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.302003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.302077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.302151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.302229] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.302351] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.302356] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38069, lrc_seqno=38069, guc_id=0, flags=0x73 in no process [-1]
<7> [243.302358] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.302426] ------------[ cut here ]------------
<4> [243.302427] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.302429] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:7/708
<4> [243.302514] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.302579]
<4> [243.302582] CPU: 4 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.302585] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.302586] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.302587] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.302593] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.302673] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.302675] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.302677] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.302679] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.302680] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.302682] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.302683] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.302685] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [243.302687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.302688] CR2: 00007fe39844b000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.302690] PKRU: 55555554
<4> [243.302691] Call Trace:
<4> [243.302692] <TASK>
<4> [243.302696] ? lock_acquire+0x60/0x300
<4> [243.302701] ? lock_release+0xd0/0x2b0
<4> [243.302706] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.302712] process_one_work+0x239/0x740
<4> [243.302719] worker_thread+0x200/0x3f0
<4> [243.302722] ? __pfx_worker_thread+0x10/0x10
<4> [243.302725] kthread+0x10d/0x150
<4> [243.302727] ? __pfx_kthread+0x10/0x10
<4> [243.302730] ret_from_fork+0x3bd/0x470
<4> [243.302732] ? __pfx_kthread+0x10/0x10
<4> [243.302735] ret_from_fork_asm+0x1a/0x30
<4> [243.302742] </TASK>
<4> [243.302744] irq event stamp: 245223
<4> [243.302745] hardirqs last enabled at (245229): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.302747] hardirqs last disabled at (245234): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.302749] softirqs last enabled at (244140): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.302752] softirqs last disabled at (244135): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.302755] ---[ end trace 0000000000000000 ]---
<7> [243.303774] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.303890] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.305383] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [243.305483] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [243.305569] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.305648] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.305741] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.305822] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.305900] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.305971] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.306042] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.306112] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.306196] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [243.306305] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [243.306411] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [243.306511] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [243.314151] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [243.398135] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.398724] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.399185] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.399610] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.400105] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.400526] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.400968] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.401356] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.401775] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.402167] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.402629] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.402813] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [243.402918] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [243.403018] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [243.448369] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38070, lrc_seqno=38070, guc_id=0, flags=0x73 in no process [-1]
<7> [243.448397] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.448939] ------------[ cut here ]------------
<4> [243.448947] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.448956] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#12: kworker/u64:8/983
<4> [243.449478] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.449957]
<4> [243.449973] CPU: 12 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.449992] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.450001] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.450010] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.450047] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.450544] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.450555] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [243.450568] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.450577] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.450586] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.450593] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.450601] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.450610] FS: 0000000000000000(0000) GS:ffff8888db283000(0000) knlGS:0000000000000000
<4> [243.450619] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.450627] CR2: 000000c0004e6000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.450636] PKRU: 55555554
<4> [243.450643] Call Trace:
<4> [243.450668] <TASK>
<4> [243.450696] ? lock_acquire+0x60/0x300
<4> [243.450730] ? lock_release+0xd0/0x2b0
<4> [243.450762] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.450803] process_one_work+0x239/0x740
<4> [243.450850] worker_thread+0x200/0x3f0
<4> [243.450869] ? __pfx_worker_thread+0x10/0x10
<4> [243.450885] kthread+0x10d/0x150
<4> [243.450898] ? __pfx_kthread+0x10/0x10
<4> [243.450917] ret_from_fork+0x3bd/0x470
<4> [243.450930] ? __pfx_kthread+0x10/0x10
<4> [243.450947] ret_from_fork_asm+0x1a/0x30
<4> [243.450999] </TASK>
<4> [243.451006] irq event stamp: 1320775
<4> [243.451013] hardirqs last enabled at (1320781): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.451028] hardirqs last disabled at (1320786): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.451040] softirqs last enabled at (1320010): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.451057] softirqs last disabled at (1320003): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.451073] ---[ end trace 0000000000000000 ]---
<6> [243.451083] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.451581] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.451620] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.453096] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.453819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.454360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.454919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.455396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.455903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.456349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.456482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.456569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.456841] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.456953] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.457046] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.458084] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.468657] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.469021] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.470283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.470472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.470659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.470826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.470990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.471153] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.471307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.471458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.471609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.471764] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.471905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.472047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.472184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.472319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.472448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.472578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.472716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.472842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.472981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.473127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.473272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.473406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.473534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.473683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.473804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.473919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.474034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.474148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.474259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.474369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.474478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.474591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.474709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.474816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.474926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.475036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.475143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.475242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.475344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.475438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.475536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.475627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.475727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.475817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.475905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.475992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.476079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.476164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.476250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.476334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.476419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.476500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.476582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.476665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.476746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.476828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.476909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.476989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.477069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.477152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.477233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.477316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.477397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.477478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.477561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.477643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.477737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.477822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.477910] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.478042] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.478047] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38070, lrc_seqno=38070, guc_id=0, flags=0x73 in no process [-1]
<7> [243.478050] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.478119] ------------[ cut here ]------------
<4> [243.478121] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.478122] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:7/708
<4> [243.478201] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.478272]
<4> [243.478274] CPU: 2 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.478278] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.478279] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.478280] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.478287] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.478363] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.478365] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.478368] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.478369] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.478370] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.478372] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.478373] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.478374] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [243.478376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.478377] CR2: 0000718d911e3000 CR3: 000000020918f000 CR4: 0000000000f52ef0
<4> [243.478379] PKRU: 55555554
<4> [243.478380] Call Trace:
<4> [243.478381] <TASK>
<4> [243.478385] ? lock_acquire+0x60/0x300
<4> [243.478391] ? lock_release+0xd0/0x2b0
<4> [243.478396] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.478402] process_one_work+0x239/0x740
<4> [243.478410] worker_thread+0x200/0x3f0
<4> [243.478413] ? __pfx_worker_thread+0x10/0x10
<4> [243.478416] kthread+0x10d/0x150
<4> [243.478418] ? __pfx_kthread+0x10/0x10
<4> [243.478421] ret_from_fork+0x3bd/0x470
<4> [243.478424] ? __pfx_kthread+0x10/0x10
<4> [243.478427] ret_from_fork_asm+0x1a/0x30
<4> [243.478435] </TASK>
<4> [243.478436] irq event stamp: 248483
<4> [243.478438] hardirqs last enabled at (248489): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.478440] hardirqs last disabled at (248494): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.478442] softirqs last enabled at (247620): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.478445] softirqs last disabled at (247609): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.478448] ---[ end trace 0000000000000000 ]---
<6> [243.478450] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.478526] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.478535] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.478947] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.479158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.479249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.479344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.479433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.479520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.479605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.479701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.479788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.479870] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.479971] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.480063] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.481098] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.491828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.492077] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.493099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.493183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.493256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.493328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.493398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.493469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.493539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.493609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.493698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.493780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.493857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.493936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.494013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.494088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.494164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.494239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.494315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.494391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.494465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.494551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.494636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.494730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.494809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.494888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.494966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.495044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.495121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.495198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.495275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.495355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.495434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.495513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.495591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.495677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.495762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.495847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.495932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.496013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.496093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.496170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.496254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.496334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.496411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.496489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.496567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.496650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.496728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.496822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.496899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.496977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.497057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.497136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.497215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.497291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.497367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.497445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.497522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.497597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.497682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.497761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.497836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.497913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.497989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.498063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.498139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.498215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.498290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.498364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.498441] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.498569] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.498573] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38070, lrc_seqno=38070, guc_id=0, flags=0x73 in no process [-1]
<7> [243.498575] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.498643] ------------[ cut here ]------------
<4> [243.498650] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.498651] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:7/708
<4> [243.498729] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.498793]
<4> [243.498795] CPU: 2 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.498798] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.498799] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.498801] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.498806] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.498882] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.498884] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.498886] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.498887] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.498889] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.498890] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.498891] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.498892] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [243.498894] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.498895] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.498897] PKRU: 55555554
<4> [243.498898] Call Trace:
<4> [243.498899] <TASK>
<4> [243.498903] ? lock_acquire+0x60/0x300
<4> [243.498908] ? lock_release+0xd0/0x2b0
<4> [243.498913] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.498919] process_one_work+0x239/0x740
<4> [243.498926] worker_thread+0x200/0x3f0
<4> [243.498929] ? __pfx_worker_thread+0x10/0x10
<4> [243.498932] kthread+0x10d/0x150
<4> [243.498934] ? __pfx_kthread+0x10/0x10
<4> [243.498937] ret_from_fork+0x3bd/0x470
<4> [243.498940] ? __pfx_kthread+0x10/0x10
<4> [243.498942] ret_from_fork_asm+0x1a/0x30
<4> [243.498950] </TASK>
<4> [243.498951] irq event stamp: 251629
<4> [243.498952] hardirqs last enabled at (251635): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.498954] hardirqs last disabled at (251640): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.498956] softirqs last enabled at (250712): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.498959] softirqs last disabled at (250707): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.498961] ---[ end trace 0000000000000000 ]---
<7> [243.500005] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.500116] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [243.506967] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38078, lrc_seqno=38078, guc_id=0, flags=0x73 in no process [-1]
<7> [243.506971] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.507060] ------------[ cut here ]------------
<4> [243.507061] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.507063] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:7/708
<4> [243.507154] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.507220]
<4> [243.507222] CPU: 8 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.507225] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.507226] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.507228] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.507234] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.507319] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.507321] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.507323] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.507325] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.507326] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.507327] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.507328] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.507330] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [243.507331] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.507333] CR2: 0000718d911e3000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [243.507334] PKRU: 55555554
<4> [243.507335] Call Trace:
<4> [243.507336] <TASK>
<4> [243.507340] ? lock_acquire+0x60/0x300
<4> [243.507345] ? lock_release+0xd0/0x2b0
<4> [243.507350] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.507356] process_one_work+0x239/0x740
<4> [243.507363] worker_thread+0x200/0x3f0
<4> [243.507366] ? __pfx_worker_thread+0x10/0x10
<4> [243.507369] kthread+0x10d/0x150
<4> [243.507371] ? __pfx_kthread+0x10/0x10
<4> [243.507374] ret_from_fork+0x3bd/0x470
<4> [243.507376] ? __pfx_kthread+0x10/0x10
<4> [243.507379] ret_from_fork_asm+0x1a/0x30
<4> [243.507387] </TASK>
<4> [243.507388] irq event stamp: 253029
<4> [243.507389] hardirqs last enabled at (253035): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.507391] hardirqs last disabled at (253040): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.507393] softirqs last enabled at (250712): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.507396] softirqs last disabled at (250707): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.507399] ---[ end trace 0000000000000000 ]---
<6> [243.507401] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.507486] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.507492] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.507919] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.508138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.508242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.508346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.508444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.508539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.508632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.508740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.508836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.508927] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.509036] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.509135] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.510178] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.520654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.520923] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.521987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.522078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.522162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.522244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.522324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.522403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.522482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.522559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.522637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.522733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.522817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.522897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.522977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.523055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.523133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.523210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.523287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.523364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.523440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.523528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.523614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.523709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.523790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.523872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.523953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.524031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.524110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.524188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.524266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.524346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.524427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.524506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.524587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.524674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.524760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.524845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.524929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.525009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.525090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.525168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.525245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.525322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.525398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.525477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.525557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.525635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.525727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.525806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.525883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.525961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.526041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.526119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.526197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.526274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.526351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.526429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.526507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.526582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.526663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.526742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.526820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.526897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.526974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.527050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.527129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.527206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.527282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.527359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.527440] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.527564] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.527569] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38078, lrc_seqno=38078, guc_id=0, flags=0x73 in no process [-1]
<7> [243.527571] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.527639] ------------[ cut here ]------------
<4> [243.527640] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.527641] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:7/708
<4> [243.527736] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.527802]
<4> [243.527804] CPU: 11 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.527808] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.527809] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.527810] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.527816] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.527892] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.527894] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.527896] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.527898] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.527899] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.527900] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.527901] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.527903] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [243.527904] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.527906] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [243.527907] PKRU: 55555554
<4> [243.527908] Call Trace:
<4> [243.527909] <TASK>
<4> [243.527913] ? lock_acquire+0x60/0x300
<4> [243.527919] ? lock_release+0xd0/0x2b0
<4> [243.527923] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.527929] process_one_work+0x239/0x740
<4> [243.527936] worker_thread+0x200/0x3f0
<4> [243.527939] ? __pfx_worker_thread+0x10/0x10
<4> [243.527942] kthread+0x10d/0x150
<4> [243.527944] ? __pfx_kthread+0x10/0x10
<4> [243.527947] ret_from_fork+0x3bd/0x470
<4> [243.527949] ? __pfx_kthread+0x10/0x10
<4> [243.527952] ret_from_fork_asm+0x1a/0x30
<4> [243.527959] </TASK>
<4> [243.527960] irq event stamp: 256429
<4> [243.527962] hardirqs last enabled at (256435): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.527964] hardirqs last disabled at (256440): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.527965] softirqs last enabled at (255330): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.527968] softirqs last disabled at (255323): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.527971] ---[ end trace 0000000000000000 ]---
<6> [243.527973] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.528052] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.528058] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.529301] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.529418] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.529531] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.529754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.529851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.529949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.530041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.530130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.530217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.530305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.530394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.530478] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.530576] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.530677] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.531974] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.542988] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.543249] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.544235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.544323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.544406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.544489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.544568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.544653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.544732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.544809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.544887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.544968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.545049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.545128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.545207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.545284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.545360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.545436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.545514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.545592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.545686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.545772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.545857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.545937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.546017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.546097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.546176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.546253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.546330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.546407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.546484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.546563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.546649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.546728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.546806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.546884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.546969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.547052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.547136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.547216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.547296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.547385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.547464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.547541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.547616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.547708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.547789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.547867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.547951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.548099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.548182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.548262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.548342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.548420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.548498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.548576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.548658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.548737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.548813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.548889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.548964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.549041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.549118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.549194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.549271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.549346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.549423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.549497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.549571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.549651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.549730] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.549854] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.549859] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38078, lrc_seqno=38078, guc_id=0, flags=0x73 in no process [-1]
<7> [243.549861] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.549929] ------------[ cut here ]------------
<4> [243.549931] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.549932] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:4/203
<4> [243.550011] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.550078]
<4> [243.550081] CPU: 3 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.550084] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.550085] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.550087] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.550093] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.550168] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.550170] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [243.550172] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.550173] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.550175] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.550176] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.550177] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.550179] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [243.550180] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.550181] CR2: 00007fe39844d000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.550183] PKRU: 55555554
<4> [243.550184] Call Trace:
<4> [243.550185] <TASK>
<4> [243.550190] ? lock_acquire+0x60/0x300
<4> [243.550195] ? lock_release+0xd0/0x2b0
<4> [243.550200] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.550206] process_one_work+0x239/0x740
<4> [243.550213] worker_thread+0x200/0x3f0
<4> [243.550216] ? __pfx_worker_thread+0x10/0x10
<4> [243.550218] kthread+0x10d/0x150
<4> [243.550221] ? __pfx_kthread+0x10/0x10
<4> [243.550224] ret_from_fork+0x3bd/0x470
<4> [243.550226] ? __pfx_kthread+0x10/0x10
<4> [243.550229] ret_from_fork_asm+0x1a/0x30
<4> [243.550236] </TASK>
<4> [243.550238] irq event stamp: 201641
<4> [243.550239] hardirqs last enabled at (201647): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.550241] hardirqs last disabled at (201652): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.550243] softirqs last enabled at (200624): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.550246] softirqs last disabled at (200617): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.550248] ---[ end trace 0000000000000000 ]---
<7> [243.551551] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.551682] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.552513] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [243.552611] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [243.552714] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.552795] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.552871] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.552945] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.553016] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.553085] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.553153] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.553223] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.553310] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [243.553420] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [243.553526] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [243.553629] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [243.564155] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [243.648127] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.648710] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.649166] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.649586] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.650065] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.650469] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.650974] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.651345] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.651745] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.652114] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.652538] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.652759] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [243.652857] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [243.652952] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [243.697903] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38079, lrc_seqno=38079, guc_id=0, flags=0x73 in no process [-1]
<7> [243.697933] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.698366] ------------[ cut here ]------------
<4> [243.698373] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.698381] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:4/203
<4> [243.698933] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.699388]
<4> [243.699403] CPU: 11 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.699419] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.699426] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.699435] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.699467] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.699949] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.699965] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [243.699986] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.699999] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.700011] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.700022] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.700034] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.700047] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [243.700062] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.700075] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.700089] PKRU: 55555554
<4> [243.700100] Call Trace:
<4> [243.700112] <TASK>
<4> [243.700148] ? lock_acquire+0x60/0x300
<4> [243.700195] ? lock_release+0xd0/0x2b0
<4> [243.700239] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.700292] process_one_work+0x239/0x740
<4> [243.700329] worker_thread+0x200/0x3f0
<4> [243.700346] ? __pfx_worker_thread+0x10/0x10
<4> [243.700361] kthread+0x10d/0x150
<4> [243.700373] ? __pfx_kthread+0x10/0x10
<4> [243.700389] ret_from_fork+0x3bd/0x470
<4> [243.700401] ? __pfx_kthread+0x10/0x10
<4> [243.700416] ret_from_fork_asm+0x1a/0x30
<4> [243.700455] </TASK>
<4> [243.700461] irq event stamp: 205017
<4> [243.700467] hardirqs last enabled at (205023): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.700480] hardirqs last disabled at (205028): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.700488] softirqs last enabled at (204272): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.700503] softirqs last disabled at (204265): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.700516] ---[ end trace 0000000000000000 ]---
<6> [243.700527] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.701149] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.701216] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.701355] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.702176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.702842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.703459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.704028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.704505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.704997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.705453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.705959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.706414] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.707007] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.707592] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.709571] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.720651] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.720968] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.722151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.722253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.722350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.722445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.722537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.722624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.722721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.722805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.722887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.722966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.723045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.723123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.723198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.723274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.723349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.723422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.723495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.723568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.723650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.723741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.723823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.723902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.723978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.724056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.724134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.724210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.724286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.724362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.724453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.724531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.724609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.724695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.724780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.724859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.724942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.725025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.725107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.725186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.725263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.725336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.725411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.725487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.725560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.725635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.725729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.725805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.725881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.725955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.726028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.726103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.726180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.726254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.726330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.726405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.726479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.726553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.726628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.726714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.726792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.726869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.726943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.727018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.727092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.727164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.727239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.727312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.727385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.727458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.727533] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.727664] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.727668] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38079, lrc_seqno=38079, guc_id=0, flags=0x73 in no process [-1]
<7> [243.727671] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.727740] ------------[ cut here ]------------
<4> [243.727741] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.727742] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:2/121
<4> [243.727814] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.727874]
<4> [243.727876] CPU: 6 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.727879] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.727880] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.727882] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.727887] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.727956] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.727958] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [243.727960] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.727961] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.727963] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.727964] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.727965] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.727966] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [243.727967] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.727969] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.727970] PKRU: 55555554
<4> [243.727971] Call Trace:
<4> [243.727972] <TASK>
<4> [243.727976] ? lock_acquire+0x60/0x300
<4> [243.727981] ? lock_release+0xd0/0x2b0
<4> [243.727985] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.727991] process_one_work+0x239/0x740
<4> [243.727998] worker_thread+0x200/0x3f0
<4> [243.728000] ? __pfx_worker_thread+0x10/0x10
<4> [243.728003] kthread+0x10d/0x150
<4> [243.728005] ? __pfx_kthread+0x10/0x10
<4> [243.728008] ret_from_fork+0x3bd/0x470
<4> [243.728010] ? __pfx_kthread+0x10/0x10
<4> [243.728012] ret_from_fork_asm+0x1a/0x30
<4> [243.728019] </TASK>
<4> [243.728020] irq event stamp: 655497
<4> [243.728021] hardirqs last enabled at (655503): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.728023] hardirqs last disabled at (655508): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.728025] softirqs last enabled at (654296): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.728027] softirqs last disabled at (654255): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.728030] ---[ end trace 0000000000000000 ]---
<6> [243.728031] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.728099] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.728107] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.728511] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.728737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.728826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.728913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.729010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.729096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.729183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.729269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.729357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.729441] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.729542] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.729637] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.730684] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.740654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.740908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.741852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.741938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.742020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.742102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.742181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.742257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.742335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.742414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.742490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.742566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.742650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.742727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.742801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.742876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.742949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.743024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.743098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.743172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.743245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.743329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.743413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.743493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.743573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.743661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.743745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.743822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.743898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.743973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.744048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.744126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.744204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.744282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.744361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.744439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.744521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.744603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.744697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.744778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.744857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.744933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.745009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.745083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.745160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.745239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.745317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.745394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.745473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.745549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.745624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.745715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.745793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.745868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.745944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.746018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.746093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.746170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.746246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.746320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.746393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.746471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.746545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.746621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.746708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.746784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.746861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.746936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.747010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.747085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.747164] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.747292] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.747296] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38079, lrc_seqno=38079, guc_id=0, flags=0x73 in no process [-1]
<7> [243.747298] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.747366] ------------[ cut here ]------------
<4> [243.747367] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.747368] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:2/121
<4> [243.747446] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.747512]
<4> [243.747514] CPU: 6 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.747517] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.747518] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.747520] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.747525] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.747601] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.747602] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [243.747605] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.747606] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.747607] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.747609] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.747610] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.747611] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [243.747613] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.747614] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.747616] PKRU: 55555554
<4> [243.747617] Call Trace:
<4> [243.747618] <TASK>
<4> [243.747622] ? lock_acquire+0x60/0x300
<4> [243.747627] ? lock_release+0xd0/0x2b0
<4> [243.747632] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.747643] process_one_work+0x239/0x740
<4> [243.747650] worker_thread+0x200/0x3f0
<4> [243.747654] ? __pfx_worker_thread+0x10/0x10
<4> [243.747656] kthread+0x10d/0x150
<4> [243.747659] ? __pfx_kthread+0x10/0x10
<4> [243.747662] ret_from_fork+0x3bd/0x470
<4> [243.747664] ? __pfx_kthread+0x10/0x10
<4> [243.747667] ret_from_fork_asm+0x1a/0x30
<4> [243.747674] </TASK>
<4> [243.747675] irq event stamp: 658649
<4> [243.747676] hardirqs last enabled at (658655): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.747679] hardirqs last disabled at (658660): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.747680] softirqs last enabled at (657380): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.747683] softirqs last disabled at (657375): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.747686] ---[ end trace 0000000000000000 ]---
<7> [243.748706] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.748822] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [243.755434] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38087, lrc_seqno=38087, guc_id=0, flags=0x73 in no process [-1]
<7> [243.755437] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.755510] ------------[ cut here ]------------
<4> [243.755511] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.755512] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:2/121
<4> [243.755593] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.755663]
<4> [243.755666] CPU: 6 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.755668] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.755670] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.755671] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.755676] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.755753] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.755755] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [243.755757] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.755759] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.755760] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.755761] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.755762] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.755763] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [243.755765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.755766] CR2: 0000718d911e3000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [243.755768] PKRU: 55555554
<4> [243.755769] Call Trace:
<4> [243.755770] <TASK>
<4> [243.755774] ? lock_acquire+0x60/0x300
<4> [243.755779] ? lock_release+0xd0/0x2b0
<4> [243.755783] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.755789] process_one_work+0x239/0x740
<4> [243.755796] worker_thread+0x200/0x3f0
<4> [243.755799] ? __pfx_worker_thread+0x10/0x10
<4> [243.755801] kthread+0x10d/0x150
<4> [243.755804] ? __pfx_kthread+0x10/0x10
<4> [243.755806] ret_from_fork+0x3bd/0x470
<4> [243.755809] ? __pfx_kthread+0x10/0x10
<4> [243.755811] ret_from_fork_asm+0x1a/0x30
<4> [243.755819] </TASK>
<4> [243.755820] irq event stamp: 659697
<4> [243.755821] hardirqs last enabled at (659703): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.755823] hardirqs last disabled at (659708): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.755825] softirqs last enabled at (659330): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.755827] softirqs last disabled at (659325): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.755830] ---[ end trace 0000000000000000 ]---
<6> [243.755832] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.755908] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.755914] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.756216] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.756439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.756540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.756646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.756768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.756859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.756946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.757034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.757123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.757207] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.757308] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.757403] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.758443] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.768648] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.768905] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.769915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.770010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.770091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.770172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.770250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.770328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.770405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.770481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.770557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.770633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.770729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.770815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.770892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.770969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.771044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.771120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.771196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.771272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.771351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.771437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.771521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.771604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.771699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.771793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.771873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.771952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.772029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.772107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.772185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.772264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.772351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.772433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.772514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.772594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.772693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.772788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.772872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.772953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.773032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.773110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.773187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.773263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.773338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.773416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.773495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.773574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.773661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.773750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.773828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.773906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.773986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.774063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.774140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.774217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.774297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.774374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.774451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.774527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.774602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.774696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.774786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.774863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.774940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.775015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.775093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.775170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.775248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.775325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.775405] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.775528] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.775532] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38087, lrc_seqno=38087, guc_id=0, flags=0x73 in no process [-1]
<7> [243.775534] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.775602] ------------[ cut here ]------------
<4> [243.775604] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.775605] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:7/708
<4> [243.775704] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.775786]
<4> [243.775789] CPU: 11 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.775792] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.775793] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.775795] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.775801] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.775877] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.775879] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.775882] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.775883] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.775884] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.775886] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.775887] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.775888] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [243.775890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.775891] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.775893] PKRU: 55555554
<4> [243.775894] Call Trace:
<4> [243.775895] <TASK>
<4> [243.775899] ? lock_acquire+0x60/0x300
<4> [243.775904] ? lock_release+0xd0/0x2b0
<4> [243.775909] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.775915] process_one_work+0x239/0x740
<4> [243.775921] worker_thread+0x200/0x3f0
<4> [243.775924] ? __pfx_worker_thread+0x10/0x10
<4> [243.775927] kthread+0x10d/0x150
<4> [243.775929] ? __pfx_kthread+0x10/0x10
<4> [243.775932] ret_from_fork+0x3bd/0x470
<4> [243.775934] ? __pfx_kthread+0x10/0x10
<4> [243.775937] ret_from_fork_asm+0x1a/0x30
<4> [243.775945] </TASK>
<4> [243.775946] irq event stamp: 260555
<4> [243.775947] hardirqs last enabled at (260561): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.775949] hardirqs last disabled at (260566): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.775951] softirqs last enabled at (259812): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.775954] softirqs last disabled at (259789): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.775956] ---[ end trace 0000000000000000 ]---
<6> [243.776651] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.776791] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.776803] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.777267] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.777380] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.777492] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.777719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.777826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.777931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.778030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.778125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.778221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.778315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.778415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.778508] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.778617] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.778728] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.779773] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.789646] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.789912] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.790936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.791029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.791115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.791201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.791285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.791368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.791451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.791534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.791616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.791706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.791788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.791869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.791950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.792031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.792112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.792193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.792274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.792354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.792434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.792526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.792616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.792716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.792803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.792890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.792975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.793057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.793139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.793219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.793300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.793385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.793471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.793554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.793642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.793726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.793815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.793905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.793993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.794079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.794162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.794244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.794327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.794409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.794490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.794572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.794658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.794741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.794823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.794918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.794999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.795081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.795166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.795248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.795330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.795411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.795491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.795573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.795661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.795742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.795824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.795908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.795990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.796072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.796154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.796234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.796318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.796404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.796485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.796567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.796654] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.796782] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.796786] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38087, lrc_seqno=38087, guc_id=0, flags=0x73 in no process [-1]
<7> [243.796789] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.796861] ------------[ cut here ]------------
<4> [243.796862] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.796864] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:7/708
<4> [243.796947] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.797013]
<4> [243.797016] CPU: 2 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.797019] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.797020] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.797021] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.797027] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.797109] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.797111] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.797113] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.797115] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.797116] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.797117] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.797118] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.797120] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [243.797122] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.797123] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.797124] PKRU: 55555554
<4> [243.797126] Call Trace:
<4> [243.797127] <TASK>
<4> [243.797131] ? lock_acquire+0x60/0x300
<4> [243.797136] ? lock_release+0xd0/0x2b0
<4> [243.797141] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.797147] process_one_work+0x239/0x740
<4> [243.797154] worker_thread+0x200/0x3f0
<4> [243.797157] ? __pfx_worker_thread+0x10/0x10
<4> [243.797160] kthread+0x10d/0x150
<4> [243.797162] ? __pfx_kthread+0x10/0x10
<4> [243.797165] ret_from_fork+0x3bd/0x470
<4> [243.797167] ? __pfx_kthread+0x10/0x10
<4> [243.797170] ret_from_fork_asm+0x1a/0x30
<4> [243.797178] </TASK>
<4> [243.797179] irq event stamp: 263709
<4> [243.797180] hardirqs last enabled at (263715): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.797183] hardirqs last disabled at (263720): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.797184] softirqs last enabled at (262528): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.797187] softirqs last disabled at (262523): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.797190] ---[ end trace 0000000000000000 ]---
<7> [243.798473] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.798595] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [243.799418] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [243.799517] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [243.799600] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.799693] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.799771] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.799845] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.799919] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [243.799991] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.800062] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.800133] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [243.800217] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [243.800327] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [243.800434] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [243.800535] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [243.814200] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [243.897897] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.898399] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.898909] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.899315] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.899743] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.900120] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.900494] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.900917] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.901167] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.901237] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.901322] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.901570] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [243.901669] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [243.901746] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.901816] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.901883] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [243.901948] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [243.902011] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [243.902074] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [243.902140] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [243.902203] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [243.902269] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [243.902366] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [243.902462] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [243.902555] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [243.948237] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38088, lrc_seqno=38088, guc_id=0, flags=0x73 in no process [-1]
<7> [243.948264] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.948773] ------------[ cut here ]------------
<4> [243.948781] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.948791] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:7/708
<4> [243.949256] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.949604]
<4> [243.949618] CPU: 3 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.949651] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.949658] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.949665] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.949696] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.950128] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.950137] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.950150] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.950158] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.950166] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.950172] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.950178] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.950185] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [243.950194] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.950201] CR2: 00007fe39844d000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.950208] PKRU: 55555554
<4> [243.950214] Call Trace:
<4> [243.950221] <TASK>
<4> [243.950243] ? lock_acquire+0x60/0x300
<4> [243.950271] ? lock_release+0xd0/0x2b0
<4> [243.950295] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.950325] process_one_work+0x239/0x740
<4> [243.950360] worker_thread+0x200/0x3f0
<4> [243.950376] ? __pfx_worker_thread+0x10/0x10
<4> [243.950390] kthread+0x10d/0x150
<4> [243.950401] ? __pfx_kthread+0x10/0x10
<4> [243.950416] ret_from_fork+0x3bd/0x470
<4> [243.950428] ? __pfx_kthread+0x10/0x10
<4> [243.950442] ret_from_fork_asm+0x1a/0x30
<4> [243.950482] </TASK>
<4> [243.950488] irq event stamp: 265133
<4> [243.950494] hardirqs last enabled at (265139): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.950506] hardirqs last disabled at (265144): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.950514] softirqs last enabled at (264370): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.950529] softirqs last disabled at (264363): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.950543] ---[ end trace 0000000000000000 ]---
<6> [243.950553] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.951031] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.951094] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.951297] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.951976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.952460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.952984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.953436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.953926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.954297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.954385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.954475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.954559] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.954663] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.954759] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.955802] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.966601] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.967006] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.968456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.968618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.968802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.968952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.969099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.969241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.969385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.969525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.969668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.969798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.969929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.970059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.970184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.970311] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.970432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.970552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.970677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.970792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.970907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.971041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.971170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.971296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.971411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.971524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.971639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.971746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.971853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.971960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.972066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.972175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.972285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.972390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.972490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.972590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.972706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.972806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.972905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.973003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.973106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.973193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.973280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.973366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.973448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.973533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.973616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.973721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.973806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.973884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.973960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.974039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.974118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.974196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.974278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.974355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.974428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.974504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.974580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.974662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.974737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.974816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.974892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.974969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.975045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.975125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.975203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.975280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.975356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.975433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.975511] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.975637] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.975642] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38088, lrc_seqno=38088, guc_id=0, flags=0x73 in no process [-1]
<7> [243.975645] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.975713] ------------[ cut here ]------------
<4> [243.975715] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.975716] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:7/708
<4> [243.975795] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.975860]
<4> [243.975863] CPU: 3 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.975866] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.975867] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.975869] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.975875] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.975950] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.975952] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.975954] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.975956] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.975957] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.975958] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.975959] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.975961] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [243.975962] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.975964] CR2: 00007fe39844d000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.975965] PKRU: 55555554
<4> [243.975966] Call Trace:
<4> [243.975968] <TASK>
<4> [243.975972] ? lock_acquire+0x60/0x300
<4> [243.975977] ? lock_release+0xd0/0x2b0
<4> [243.975982] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.975988] process_one_work+0x239/0x740
<4> [243.975995] worker_thread+0x200/0x3f0
<4> [243.975998] ? __pfx_worker_thread+0x10/0x10
<4> [243.976000] kthread+0x10d/0x150
<4> [243.976003] ? __pfx_kthread+0x10/0x10
<4> [243.976006] ret_from_fork+0x3bd/0x470
<4> [243.976008] ? __pfx_kthread+0x10/0x10
<4> [243.976011] ret_from_fork_asm+0x1a/0x30
<4> [243.976018] </TASK>
<4> [243.976019] irq event stamp: 268325
<4> [243.976020] hardirqs last enabled at (268331): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.976023] hardirqs last disabled at (268336): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.976024] softirqs last enabled at (267402): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.976027] softirqs last disabled at (267395): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.976030] ---[ end trace 0000000000000000 ]---
<6> [243.976032] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [243.976107] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [243.976113] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [243.976519] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [243.976734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [243.976824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [243.976918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [243.977009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [243.977097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [243.977183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [243.977269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [243.977357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [243.977441] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [243.977541] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [243.977641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [243.978682] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [243.989412] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [243.989664] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [243.990619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [243.990719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [243.990801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [243.990882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [243.990962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [243.991038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [243.991118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [243.991197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [243.991275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [243.991352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [243.991428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [243.991503] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [243.991580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [243.991662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [243.991738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [243.991813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [243.991889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [243.991964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [243.992039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [243.992125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [243.992209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [243.992290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [243.992368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [243.992449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [243.992531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [243.992610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [243.992704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [243.992782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [243.992859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [243.992942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [243.993022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [243.993101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [243.993180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [243.993258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [243.993342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [243.993425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [243.993508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [243.993588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [243.993676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [243.993752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [243.993830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [243.993911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [243.993986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [243.994064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [243.994140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [243.994221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [243.994306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [243.994386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [243.994464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [243.994542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [243.994622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [243.994711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [243.994790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [243.994867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [243.994944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [243.995019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [243.995096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [243.995171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [243.995246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [243.995323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [243.995401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [243.995480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [243.995557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [243.995636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [243.995713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [243.995791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [243.995867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [243.995942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [243.996020] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [243.996141] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [243.996145] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38088, lrc_seqno=38088, guc_id=0, flags=0x73 in no process [-1]
<7> [243.996147] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [243.996216] ------------[ cut here ]------------
<4> [243.996217] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [243.996218] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:7/708
<4> [243.996297] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [243.996360]
<4> [243.996362] CPU: 3 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [243.996365] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [243.996366] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [243.996368] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [243.996373] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [243.996447] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [243.996449] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [243.996451] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [243.996452] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [243.996453] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [243.996455] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [243.996456] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [243.996457] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [243.996459] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [243.996460] CR2: 00007fe39844d000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [243.996461] PKRU: 55555554
<4> [243.996463] Call Trace:
<4> [243.996464] <TASK>
<4> [243.996467] ? lock_acquire+0x60/0x300
<4> [243.996474] ? lock_release+0xd0/0x2b0
<4> [243.996479] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [243.996485] process_one_work+0x239/0x740
<4> [243.996492] worker_thread+0x200/0x3f0
<4> [243.996495] ? __pfx_worker_thread+0x10/0x10
<4> [243.996497] kthread+0x10d/0x150
<4> [243.996500] ? __pfx_kthread+0x10/0x10
<4> [243.996502] ret_from_fork+0x3bd/0x470
<4> [243.996504] ? __pfx_kthread+0x10/0x10
<4> [243.996507] ret_from_fork_asm+0x1a/0x30
<4> [243.996515] </TASK>
<4> [243.996516] irq event stamp: 271473
<4> [243.996517] hardirqs last enabled at (271479): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [243.996519] hardirqs last disabled at (271484): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [243.996521] softirqs last enabled at (270420): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.996523] softirqs last disabled at (270413): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [243.996526] ---[ end trace 0000000000000000 ]---
<7> [243.997564] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [243.997685] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [244.003381] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38096, lrc_seqno=38096, guc_id=0, flags=0x73 in no process [-1]
<7> [244.003386] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.003464] ------------[ cut here ]------------
<4> [244.003465] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.003467] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:7/708
<4> [244.003553] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.003618]
<4> [244.003621] CPU: 10 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.003624] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.003625] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.003626] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.003644] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.003723] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.003725] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [244.003727] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.003729] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.003730] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.003731] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.003733] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.003734] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [244.003735] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.003737] CR2: 00007fe398451000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.003738] PKRU: 55555554
<4> [244.003739] Call Trace:
<4> [244.003741] <TASK>
<4> [244.003744] ? lock_acquire+0x60/0x300
<4> [244.003749] ? lock_release+0xd0/0x2b0
<4> [244.003754] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.003760] process_one_work+0x239/0x740
<4> [244.003766] worker_thread+0x200/0x3f0
<4> [244.003769] ? __pfx_worker_thread+0x10/0x10
<4> [244.003772] kthread+0x10d/0x150
<4> [244.003774] ? __pfx_kthread+0x10/0x10
<4> [244.003777] ret_from_fork+0x3bd/0x470
<4> [244.003779] ? __pfx_kthread+0x10/0x10
<4> [244.003782] ret_from_fork_asm+0x1a/0x30
<4> [244.003789] </TASK>
<4> [244.003790] irq event stamp: 272721
<4> [244.003792] hardirqs last enabled at (272727): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.003794] hardirqs last disabled at (272732): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.003795] softirqs last enabled at (272556): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.003798] softirqs last disabled at (272537): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.003801] ---[ end trace 0000000000000000 ]---
<6> [244.003803] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.003881] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.003887] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.004316] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.004528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.004626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.004742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.004835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.004926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.005013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.005101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.005194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.005281] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.005383] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.005477] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.006544] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.016642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.016903] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.017967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.018054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.018136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.018216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.018296] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.018375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.018453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.018529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.018607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.018696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.018773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.018847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.018925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.018999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.019074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.019148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.019221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.019298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.019372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.019459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.019544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.019627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.019724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.019805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.019883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.019961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.020038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.020114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.020189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.020268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.020346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.020424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.020505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.020585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.020680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.020765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.020851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.020934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.021016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.021093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.021172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.021248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.021324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.021407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.021486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.021564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.021647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.021729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.021808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.021887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.021968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.022045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.022126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.022203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.022282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.022360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.022436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.022511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.022585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.022672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.022747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.022823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.022899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.022972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.023048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.023123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.023198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.023273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.023351] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.023474] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.023478] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38096, lrc_seqno=38096, guc_id=0, flags=0x73 in no process [-1]
<7> [244.023480] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.023548] ------------[ cut here ]------------
<4> [244.023549] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.023550] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:7/708
<4> [244.023628] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.023697]
<4> [244.023700] CPU: 10 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.023703] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.023704] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.023705] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.023711] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.023786] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.023788] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [244.023790] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.023792] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.023793] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.023794] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.023795] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.023797] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [244.023798] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.023800] CR2: 00007fe398451000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.023801] PKRU: 55555554
<4> [244.023802] Call Trace:
<4> [244.023803] <TASK>
<4> [244.023807] ? lock_acquire+0x60/0x300
<4> [244.023812] ? lock_release+0xd0/0x2b0
<4> [244.023817] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.023823] process_one_work+0x239/0x740
<4> [244.023829] worker_thread+0x200/0x3f0
<4> [244.023832] ? __pfx_worker_thread+0x10/0x10
<4> [244.023835] kthread+0x10d/0x150
<4> [244.023837] ? __pfx_kthread+0x10/0x10
<4> [244.023840] ret_from_fork+0x3bd/0x470
<4> [244.023842] ? __pfx_kthread+0x10/0x10
<4> [244.023844] ret_from_fork_asm+0x1a/0x30
<4> [244.023852] </TASK>
<4> [244.023853] irq event stamp: 275879
<4> [244.023854] hardirqs last enabled at (275885): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.023856] hardirqs last disabled at (275890): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.023858] softirqs last enabled at (274604): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.023860] softirqs last disabled at (274599): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.023863] ---[ end trace 0000000000000000 ]---
<6> [244.023864] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.023942] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.023949] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.025189] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.025304] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.025415] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.025648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.025753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.025856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.025949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.026040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.026127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.026215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.026304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.026389] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.026490] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.026584] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.027646] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.037641] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.037901] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.038857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.038945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.039026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.039105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.039184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.039261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.039339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.039416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.039492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.039569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.039651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.039728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.039805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.039882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.039959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.040035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.040110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.040185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.040261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.040348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.040433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.040515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.040595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.040688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.040769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.040848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.040926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.041003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.041079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.041160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.041241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.041319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.041398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.041477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.041564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.041659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.041746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.041831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.041914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.041992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.042069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.042146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.042223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.042301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.042378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.042454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.042529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.042604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.042692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.042774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.042860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.042946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.043026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.043104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.043181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.043259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.043336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.043411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.043555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.043671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.043753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.043831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.043908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.043987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.044066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.044144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.044220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.044296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.044378] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.044503] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.044508] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38096, lrc_seqno=38096, guc_id=0, flags=0x73 in no process [-1]
<7> [244.044510] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.044578] ------------[ cut here ]------------
<4> [244.044579] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.044581] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#7: kworker/u64:2/121
<4> [244.044671] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.044738]
<4> [244.044741] CPU: 7 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.044744] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.044745] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.044747] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.044752] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.044829] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.044831] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.044833] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.044835] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.044836] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.044838] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.044839] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.044840] FS: 0000000000000000(0000) GS:ffff8888db003000(0000) knlGS:0000000000000000
<4> [244.044842] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.044843] CR2: 0000718d911e3000 CR3: 000000020918f000 CR4: 0000000000f52ef0
<4> [244.044845] PKRU: 55555554
<4> [244.044846] Call Trace:
<4> [244.044847] <TASK>
<4> [244.044851] ? lock_acquire+0x60/0x300
<4> [244.044856] ? lock_release+0xd0/0x2b0
<4> [244.044861] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.044867] process_one_work+0x239/0x740
<4> [244.044875] worker_thread+0x200/0x3f0
<4> [244.044878] ? __pfx_worker_thread+0x10/0x10
<4> [244.044881] kthread+0x10d/0x150
<4> [244.044883] ? __pfx_kthread+0x10/0x10
<4> [244.044886] ret_from_fork+0x3bd/0x470
<4> [244.044888] ? __pfx_kthread+0x10/0x10
<4> [244.044891] ret_from_fork_asm+0x1a/0x30
<4> [244.044898] </TASK>
<4> [244.044899] irq event stamp: 663269
<4> [244.044901] hardirqs last enabled at (663275): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.044903] hardirqs last disabled at (663280): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.044904] softirqs last enabled at (662188): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.044907] softirqs last disabled at (662179): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.044910] ---[ end trace 0000000000000000 ]---
<7> [244.045933] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.046046] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.047629] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [244.047749] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [244.047841] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.047926] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048007] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048085] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048162] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.048237] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048311] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048386] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.048479] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [244.048689] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [244.048779] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [244.048860] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.048938] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049016] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049093] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049169] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.049243] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049316] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049390] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.049471] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [244.049586] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [244.049706] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [244.049814] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [244.064136] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [244.148142] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.148748] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.149214] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.149682] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.150105] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.150515] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.150962] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.151362] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.151805] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.152221] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.152592] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.152788] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.152863] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.152932] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.153001] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.153071] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.153137] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.153204] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.153270] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.153335] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.153401] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.153469] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.153575] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [244.153688] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [244.153795] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [244.197878] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38097, lrc_seqno=38097, guc_id=0, flags=0x73 in no process [-1]
<7> [244.197902] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.198295] ------------[ cut here ]------------
<4> [244.198302] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.198309] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:7/708
<4> [244.198800] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.199305]
<4> [244.199320] CPU: 11 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.199338] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.199345] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.199354] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.199385] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.199865] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.199875] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [244.199889] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.199898] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.199906] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.199913] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.199920] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.199928] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [244.199937] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.199945] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.199954] PKRU: 55555554
<4> [244.199962] Call Trace:
<4> [244.199968] <TASK>
<4> [244.199989] ? lock_acquire+0x60/0x300
<4> [244.200018] ? lock_release+0xd0/0x2b0
<4> [244.200043] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.200074] process_one_work+0x239/0x740
<4> [244.200109] worker_thread+0x200/0x3f0
<4> [244.200126] ? __pfx_worker_thread+0x10/0x10
<4> [244.200140] kthread+0x10d/0x150
<4> [244.200152] ? __pfx_kthread+0x10/0x10
<4> [244.200168] ret_from_fork+0x3bd/0x470
<4> [244.200181] ? __pfx_kthread+0x10/0x10
<4> [244.200196] ret_from_fork_asm+0x1a/0x30
<4> [244.200236] </TASK>
<4> [244.200242] irq event stamp: 277687
<4> [244.200248] hardirqs last enabled at (277693): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.200259] hardirqs last disabled at (277698): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.200269] softirqs last enabled at (276942): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.200285] softirqs last disabled at (276937): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.200299] ---[ end trace 0000000000000000 ]---
<6> [244.200309] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.200780] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.200838] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.201005] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.201761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.202303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.203019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.203498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.203980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.204404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.204866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.205298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.205731] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.206234] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.206731] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.208543] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.219699] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.220048] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.221218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.221338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.221445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.221544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.221647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.221740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.221832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.221922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.222008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.222095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.222179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.222265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.222349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.222430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.222511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.222592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.222688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.222773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.222856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.222949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.223039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.223125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.223217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.223306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.223393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.223478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.223561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.223649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.223733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.223819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.223905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.223988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.224074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.224159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.224249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.224339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.224429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.224515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.224600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.224690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.224773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.224855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.224936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.225018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.225102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.225186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.225274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.225358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.225442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.225526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.225617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.225709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.225793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.225875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.225956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.226039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.226122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.226202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.226282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.226365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.226446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.226529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.226614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.226705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.226788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.226870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.226950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.227031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.227114] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.227244] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.227249] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38097, lrc_seqno=38097, guc_id=0, flags=0x73 in no process [-1]
<7> [244.227251] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.227324] ------------[ cut here ]------------
<4> [244.227326] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.227327] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:2/121
<4> [244.227411] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.227475]
<4> [244.227478] CPU: 4 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.227481] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.227482] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.227484] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.227491] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.227573] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.227575] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.227578] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.227579] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.227581] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.227582] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.227584] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.227585] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [244.227587] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.227589] CR2: 00007fe39844b000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.227591] PKRU: 55555554
<4> [244.227592] Call Trace:
<4> [244.227593] <TASK>
<4> [244.227598] ? lock_acquire+0x60/0x300
<4> [244.227603] ? lock_release+0xd0/0x2b0
<4> [244.227608] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.227615] process_one_work+0x239/0x740
<4> [244.227622] worker_thread+0x200/0x3f0
<4> [244.227625] ? __pfx_worker_thread+0x10/0x10
<4> [244.227628] kthread+0x10d/0x150
<4> [244.227634] ? __pfx_kthread+0x10/0x10
<4> [244.227638] ret_from_fork+0x3bd/0x470
<4> [244.227640] ? __pfx_kthread+0x10/0x10
<4> [244.227643] ret_from_fork_asm+0x1a/0x30
<4> [244.227651] </TASK>
<4> [244.227652] irq event stamp: 666443
<4> [244.227654] hardirqs last enabled at (666449): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.227656] hardirqs last disabled at (666454): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.227658] softirqs last enabled at (665298): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.227662] softirqs last disabled at (665291): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.227664] ---[ end trace 0000000000000000 ]---
<6> [244.228585] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.228697] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.228705] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.228679] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.228802] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.228919] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.229142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.229249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.229355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.229456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.229557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.229658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.229753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.229846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.229934] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.230041] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.230142] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.231184] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.241639] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.241901] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.242996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.243085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.243171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.243256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.243340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.243424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.243511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.243593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.243705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.243788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.243869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.243951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.244033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.244116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.244197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.244278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.244359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.244439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.244517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.244608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.244707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.244794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.244879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.244967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.245055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.245137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.245219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.245301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.245382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.245467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.245550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.245636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.245719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.245803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.245892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.245980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.246069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.246155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.246239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.246322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.246404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.246484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.246564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.246651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.246734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.246816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.246898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.246980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.247059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.247142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.247226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.247306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.247388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.247470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.247550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.247638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.247720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.247800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.247879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.247961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.248042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.248125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.248205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.248283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.248364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.248445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.248524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.248603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.248695] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.248825] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.248830] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38097, lrc_seqno=38097, guc_id=0, flags=0x73 in no process [-1]
<7> [244.248832] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.248904] ------------[ cut here ]------------
<4> [244.248906] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.248907] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:2/121
<4> [244.248991] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.249056]
<4> [244.249058] CPU: 8 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.249061] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.249062] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.249063] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.249069] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.249151] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.249153] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.249155] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.249156] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.249158] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.249159] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.249160] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.249161] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [244.249163] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.249164] CR2: 00007fe398453000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [244.249166] PKRU: 55555554
<4> [244.249167] Call Trace:
<4> [244.249168] <TASK>
<4> [244.249172] ? lock_acquire+0x60/0x300
<4> [244.249178] ? lock_release+0xd0/0x2b0
<4> [244.249182] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.249189] process_one_work+0x239/0x740
<4> [244.249196] worker_thread+0x200/0x3f0
<4> [244.249199] ? __pfx_worker_thread+0x10/0x10
<4> [244.249202] kthread+0x10d/0x150
<4> [244.249204] ? __pfx_kthread+0x10/0x10
<4> [244.249207] ret_from_fork+0x3bd/0x470
<4> [244.249209] ? __pfx_kthread+0x10/0x10
<4> [244.249212] ret_from_fork_asm+0x1a/0x30
<4> [244.249220] </TASK>
<4> [244.249221] irq event stamp: 669715
<4> [244.249222] hardirqs last enabled at (669721): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.249224] hardirqs last disabled at (669726): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.249226] softirqs last enabled at (668530): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.249229] softirqs last disabled at (668523): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.249231] ---[ end trace 0000000000000000 ]---
<7> [244.250536] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.250671] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [244.256269] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38105, lrc_seqno=38105, guc_id=0, flags=0x73 in no process [-1]
<7> [244.256272] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.256349] ------------[ cut here ]------------
<4> [244.256350] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.256352] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:2/121
<4> [244.256437] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.256501]
<4> [244.256504] CPU: 8 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.256507] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.256508] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.256509] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.256515] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.256597] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.256599] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.256601] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.256602] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.256604] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.256605] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.256606] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.256607] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [244.256609] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.256610] CR2: 00007fe398453000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [244.256612] PKRU: 55555554
<4> [244.256613] Call Trace:
<4> [244.256614] <TASK>
<4> [244.256618] ? lock_acquire+0x60/0x300
<4> [244.256623] ? lock_release+0xd0/0x2b0
<4> [244.256628] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.256639] process_one_work+0x239/0x740
<4> [244.256646] worker_thread+0x200/0x3f0
<4> [244.256649] ? __pfx_worker_thread+0x10/0x10
<4> [244.256652] kthread+0x10d/0x150
<4> [244.256655] ? __pfx_kthread+0x10/0x10
<4> [244.256658] ret_from_fork+0x3bd/0x470
<4> [244.256660] ? __pfx_kthread+0x10/0x10
<4> [244.256663] ret_from_fork_asm+0x1a/0x30
<4> [244.256671] </TASK>
<4> [244.256672] irq event stamp: 670879
<4> [244.256673] hardirqs last enabled at (670885): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.256675] hardirqs last disabled at (670890): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.256677] softirqs last enabled at (668530): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.256680] softirqs last disabled at (668523): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.256682] ---[ end trace 0000000000000000 ]---
<6> [244.256684] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.256765] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.256772] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.257000] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.257215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.257315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.257416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.257514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.257608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.257708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.257800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.257892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.257980] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.258086] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.258185] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.259224] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.269639] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.269902] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.270901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.270995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.271079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.271164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.271249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.271332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.271414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.271496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.271579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.271675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.271758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.271840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.271921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.272000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.272080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.272161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.272243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.272322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.272404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.272495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.272585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.272680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.272766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.272852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.272938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.273022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.273105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.273188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.273271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.273354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.273440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.273524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.273607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.273724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.273821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.273914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.274005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.274095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.274186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.274271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.274356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.274440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.274521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.274605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.274699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.274783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.274867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.274950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.275030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.275113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.275198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.275283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.275366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.275448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.275530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.275611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.275705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.275787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.275866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.275949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.276031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.276113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.276194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.276275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.276357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.276438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.276519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.276600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.276693] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.276819] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.276823] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38105, lrc_seqno=38105, guc_id=0, flags=0x73 in no process [-1]
<7> [244.276825] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.276897] ------------[ cut here ]------------
<4> [244.276898] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.276899] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:2/121
<4> [244.276983] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.277047]
<4> [244.277049] CPU: 8 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.277052] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.277054] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.277055] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.277061] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.277143] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.277145] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.277147] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.277148] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.277150] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.277151] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.277152] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.277154] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [244.277155] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.277157] CR2: 00007fe398453000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [244.277158] PKRU: 55555554
<4> [244.277159] Call Trace:
<4> [244.277160] <TASK>
<4> [244.277164] ? lock_acquire+0x60/0x300
<4> [244.277169] ? lock_release+0xd0/0x2b0
<4> [244.277174] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.277180] process_one_work+0x239/0x740
<4> [244.277186] worker_thread+0x200/0x3f0
<4> [244.277189] ? __pfx_worker_thread+0x10/0x10
<4> [244.277192] kthread+0x10d/0x150
<4> [244.277194] ? __pfx_kthread+0x10/0x10
<4> [244.277197] ret_from_fork+0x3bd/0x470
<4> [244.277199] ? __pfx_kthread+0x10/0x10
<4> [244.277202] ret_from_fork_asm+0x1a/0x30
<4> [244.277210] </TASK>
<4> [244.277211] irq event stamp: 674011
<4> [244.277212] hardirqs last enabled at (674017): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.277214] hardirqs last disabled at (674022): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.277215] softirqs last enabled at (672902): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.277218] softirqs last disabled at (672895): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.277221] ---[ end trace 0000000000000000 ]---
<6> [244.277223] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.277303] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.277308] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.278800] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.278927] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.279387] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.279609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.279720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.279824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.279921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.280015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.280106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.280197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.280293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.280383] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.280488] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.280588] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.281633] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.291640] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.291899] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.292853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.292946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.293030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.293116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.293200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.293283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.293366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.293448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.293529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.293610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.293704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.293788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.293870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.293951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.294032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.294113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.294194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.294274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.294354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.294443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.294532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.294619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.294717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.294803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.294886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.294967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.295048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.295129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.295211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.295296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.295380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.295464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.295549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.295638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.295728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.295818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.295908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.295994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.296080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.296162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.296245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.296326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.296406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.296488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.296572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.296657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.296739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.296822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.296904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.296987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.297072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.297154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.297237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.297318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.297399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.297482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.297564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.297647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.297727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.297811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.297891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.297973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.298053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.298135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.298217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.298299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.298379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.298459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.298542] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.298908] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.298913] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38105, lrc_seqno=38105, guc_id=0, flags=0x73 in no process [-1]
<7> [244.298916] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.298990] ------------[ cut here ]------------
<4> [244.298991] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.298993] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:2/121
<4> [244.299076] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.299140]
<4> [244.299142] CPU: 8 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.299145] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.299147] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.299148] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.299154] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.299236] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.299237] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.299239] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.299241] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.299242] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.299243] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.299245] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.299246] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [244.299247] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.299249] CR2: 00007fe398453000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.299250] PKRU: 55555554
<4> [244.299251] Call Trace:
<4> [244.299252] <TASK>
<4> [244.299256] ? lock_acquire+0x60/0x300
<4> [244.299262] ? lock_release+0xd0/0x2b0
<4> [244.299266] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.299272] process_one_work+0x239/0x740
<4> [244.299280] worker_thread+0x200/0x3f0
<4> [244.299283] ? __pfx_worker_thread+0x10/0x10
<4> [244.299286] kthread+0x10d/0x150
<4> [244.299288] ? __pfx_kthread+0x10/0x10
<4> [244.299291] ret_from_fork+0x3bd/0x470
<4> [244.299293] ? __pfx_kthread+0x10/0x10
<4> [244.299295] ret_from_fork_asm+0x1a/0x30
<4> [244.299303] </TASK>
<4> [244.299304] irq event stamp: 677197
<4> [244.299305] hardirqs last enabled at (677203): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.299308] hardirqs last disabled at (677208): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.299309] softirqs last enabled at (676024): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.299312] softirqs last disabled at (676015): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.299315] ---[ end trace 0000000000000000 ]---
<7> [244.300585] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.300716] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.301532] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [244.301648] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [244.301744] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.301828] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.301907] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.301985] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.302060] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.302133] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.302207] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.302281] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.302370] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [244.302487] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [244.302600] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [244.302723] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [244.313884] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [244.398130] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.398721] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.399176] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.399596] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.400118] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.400659] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.401061] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.401446] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.401882] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.402263] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.402676] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.402790] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [244.402893] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [244.402991] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [244.448357] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38106, lrc_seqno=38106, guc_id=0, flags=0x73 in no process [-1]
<7> [244.448385] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.448927] ------------[ cut here ]------------
<4> [244.448936] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.448945] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#12: kworker/u64:4/203
<4> [244.449468] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.449890]
<4> [244.449906] CPU: 12 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.449925] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.449933] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.449943] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.449981] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.450479] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.450489] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [244.450503] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.450512] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.450520] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.450528] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.450536] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.450544] FS: 0000000000000000(0000) GS:ffff8888db283000(0000) knlGS:0000000000000000
<4> [244.450554] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.450562] CR2: 000000c0004e6000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.450571] PKRU: 55555554
<4> [244.450577] Call Trace:
<4> [244.450585] <TASK>
<4> [244.450613] ? lock_acquire+0x60/0x300
<4> [244.450666] ? lock_release+0xd0/0x2b0
<4> [244.450699] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.450739] process_one_work+0x239/0x740
<4> [244.450785] worker_thread+0x200/0x3f0
<4> [244.450805] ? __pfx_worker_thread+0x10/0x10
<4> [244.450821] kthread+0x10d/0x150
<4> [244.450835] ? __pfx_kthread+0x10/0x10
<4> [244.450855] ret_from_fork+0x3bd/0x470
<4> [244.450869] ? __pfx_kthread+0x10/0x10
<4> [244.450886] ret_from_fork_asm+0x1a/0x30
<4> [244.450937] </TASK>
<4> [244.450944] irq event stamp: 208375
<4> [244.450951] hardirqs last enabled at (208381): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.450965] hardirqs last disabled at (208386): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.450976] softirqs last enabled at (207608): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.450992] softirqs last disabled at (207601): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.451007] ---[ end trace 0000000000000000 ]---
<6> [244.451126] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.451710] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.451751] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.452053] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.452721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.453331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.453928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.454390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.454868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.455295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.455739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.456178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.456275] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.456376] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.456471] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.457506] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.468329] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.468761] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.470059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.470256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.470437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.470615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.470818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.470998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.471175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.471345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.471520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.471708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.471875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.472037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.472192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.472349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.472504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.472661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.472814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.472966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.473113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.473279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.473434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.473590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.473746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.473887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.474028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.474159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.474286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.474410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.474534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.474667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.474797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.474923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.475045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.475171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.475293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.475412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.475529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.475645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.475754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.475860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.475963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.476065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.476163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.476262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.476362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.476458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.476552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.476651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.476746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.476840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.476933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.477024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.477113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.477202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.477287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.477372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.477457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.477541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.477625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.477723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.477806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.477888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.477970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.478051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.478133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.478213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.478292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.478371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.478454] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.478582] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.478587] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38106, lrc_seqno=38106, guc_id=0, flags=0x73 in no process [-1]
<7> [244.478590] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.478672] ------------[ cut here ]------------
<4> [244.478673] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.478675] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [244.478759] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.478828]
<4> [244.478830] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.478834] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.478835] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.478837] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.478844] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.478925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.478927] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [244.478930] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.478931] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.478933] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.478934] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.478935] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.478936] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [244.478938] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.478939] CR2: 00007fe39844b000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.478941] PKRU: 55555554
<4> [244.478942] Call Trace:
<4> [244.478943] <TASK>
<4> [244.478948] ? lock_acquire+0x60/0x300
<4> [244.478954] ? lock_release+0xd0/0x2b0
<4> [244.478959] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.478965] process_one_work+0x239/0x740
<4> [244.478973] worker_thread+0x200/0x3f0
<4> [244.478976] ? __pfx_worker_thread+0x10/0x10
<4> [244.478979] kthread+0x10d/0x150
<4> [244.478982] ? __pfx_kthread+0x10/0x10
<4> [244.478985] ret_from_fork+0x3bd/0x470
<4> [244.478988] ? __pfx_kthread+0x10/0x10
<4> [244.478990] ret_from_fork_asm+0x1a/0x30
<4> [244.478999] </TASK>
<4> [244.479000] irq event stamp: 211543
<4> [244.479001] hardirqs last enabled at (211549): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.479004] hardirqs last disabled at (211554): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.479005] softirqs last enabled at (210264): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.479009] softirqs last disabled at (210257): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.479012] ---[ end trace 0000000000000000 ]---
<6> [244.479013] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.479096] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.479103] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.480342] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.480464] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.480580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.480826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.480929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.481030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.481123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.481212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.481301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.481389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.481479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.481563] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.481670] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.481766] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.482800] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.492640] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.492897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.493922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.494009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.494090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.494172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.494253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.494333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.494410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.494487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.494564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.494652] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.494729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.494803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.494878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.494955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.495032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.495109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.495185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.495261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.495337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.495423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.495506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.495589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.495678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.495759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.495837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.495914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.495991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.496070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.496150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.496230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.496307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.496386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.496465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.496544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.496628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.496726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.496811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.496893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.496975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.497052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.497130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.497207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.497282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.497359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.497440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.497517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.497598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.497683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.497762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.497841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.497921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.497996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.498073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.498149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.498223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.498299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.498375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.498451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.498525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.498604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.498687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.498764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.498840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.498915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.498993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.499070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.499145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.499221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.499300] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.499423] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.499427] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38106, lrc_seqno=38106, guc_id=0, flags=0x73 in no process [-1]
<7> [244.499430] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.499498] ------------[ cut here ]------------
<4> [244.499499] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.499500] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:7/708
<4> [244.499589] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.499660]
<4> [244.499663] CPU: 11 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.499666] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.499667] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.499669] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.499675] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.499753] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.499755] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [244.499757] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.499758] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.499760] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.499761] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.499762] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.499763] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [244.499765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.499766] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.499768] PKRU: 55555554
<4> [244.499769] Call Trace:
<4> [244.499770] <TASK>
<4> [244.499774] ? lock_acquire+0x60/0x300
<4> [244.499779] ? lock_release+0xd0/0x2b0
<4> [244.499784] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.499790] process_one_work+0x239/0x740
<4> [244.499797] worker_thread+0x200/0x3f0
<4> [244.499800] ? __pfx_worker_thread+0x10/0x10
<4> [244.499803] kthread+0x10d/0x150
<4> [244.499805] ? __pfx_kthread+0x10/0x10
<4> [244.499808] ret_from_fork+0x3bd/0x470
<4> [244.499811] ? __pfx_kthread+0x10/0x10
<4> [244.499813] ret_from_fork_asm+0x1a/0x30
<4> [244.499821] </TASK>
<4> [244.499822] irq event stamp: 283353
<4> [244.499823] hardirqs last enabled at (283359): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.499826] hardirqs last disabled at (283364): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.499827] softirqs last enabled at (282178): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.499830] softirqs last disabled at (282169): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.499833] ---[ end trace 0000000000000000 ]---
<7> [244.501074] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.501190] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [244.507584] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38114, lrc_seqno=38114, guc_id=0, flags=0x73 in no process [-1]
<7> [244.507589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.507689] ------------[ cut here ]------------
<4> [244.507690] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.507692] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#8: kworker/u64:7/708
<4> [244.507784] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.507850]
<4> [244.507853] CPU: 8 UID: 0 PID: 708 Comm: kworker/u64:7 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.507856] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.507857] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.507858] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.507864] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.507951] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.507952] RSP: 0018:ffffc900016c3ca0 EFLAGS: 00010246
<4> [244.507955] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.507956] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.507958] RBP: ffffc900016c3db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.507959] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.507960] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.507961] FS: 0000000000000000(0000) GS:ffff8888db083000(0000) knlGS:0000000000000000
<4> [244.507963] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.507964] CR2: 00007fe398453000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.507966] PKRU: 55555554
<4> [244.507967] Call Trace:
<4> [244.507968] <TASK>
<4> [244.507972] ? lock_acquire+0x60/0x300
<4> [244.507978] ? lock_release+0xd0/0x2b0
<4> [244.507983] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.507989] process_one_work+0x239/0x740
<4> [244.507996] worker_thread+0x200/0x3f0
<4> [244.507999] ? __pfx_worker_thread+0x10/0x10
<4> [244.508002] kthread+0x10d/0x150
<4> [244.508004] ? __pfx_kthread+0x10/0x10
<4> [244.508007] ret_from_fork+0x3bd/0x470
<4> [244.508010] ? __pfx_kthread+0x10/0x10
<4> [244.508012] ret_from_fork_asm+0x1a/0x30
<4> [244.508020] </TASK>
<4> [244.508021] irq event stamp: 284495
<4> [244.508022] hardirqs last enabled at (284501): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.508025] hardirqs last disabled at (284506): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.508026] softirqs last enabled at (283734): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.508029] softirqs last disabled at (283727): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.508032] ---[ end trace 0000000000000000 ]---
<6> [244.508034] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.508118] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.508124] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.508427] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.508645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.508771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.508873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.508966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.509054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.509141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.509227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.509315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.509397] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.509498] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.509591] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.510648] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.520656] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.520915] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.521924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.522009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.522091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.522172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.522251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.522329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.522404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.522479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.522553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.522629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.522724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.522817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.522895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.522972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.523049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.523126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.523200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.523277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.523350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.523435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.523520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.523600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.523693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.523795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.523877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.523959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.524037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.524115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.524192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.524270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.524349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.524426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.524503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.524581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.524677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.524788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.524878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.524961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.525042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.525120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.525198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.525275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.525351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.525430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.525509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.525586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.525675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.525774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.525852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.525931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.526012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.526089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.526167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.526245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.526320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.526397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.526474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.526549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.526625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.526720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.526813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.526892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.526969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.527045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.527122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.527200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.527278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.527355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.527450] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.527575] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.527579] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38114, lrc_seqno=38114, guc_id=0, flags=0x73 in no process [-1]
<7> [244.527582] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.527829] ------------[ cut here ]------------
<4> [244.527831] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.527834] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [244.528192] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.528324]
<4> [244.528328] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.528336] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.528339] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.528341] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.528360] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.528552] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.528554] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [244.528556] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.528558] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.528559] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.528560] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.528561] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.528563] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [244.528564] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.528565] CR2: 00007fe398454000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.528567] PKRU: 55555554
<4> [244.528568] Call Trace:
<4> [244.528569] <TASK>
<4> [244.528573] ? lock_acquire+0x60/0x300
<4> [244.528580] ? lock_release+0xd0/0x2b0
<4> [244.528587] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.528596] process_one_work+0x239/0x740
<4> [244.528608] worker_thread+0x200/0x3f0
<4> [244.528613] ? __pfx_worker_thread+0x10/0x10
<4> [244.528618] kthread+0x10d/0x150
<4> [244.528621] ? __pfx_kthread+0x10/0x10
<4> [244.528626] ret_from_fork+0x3bd/0x470
<4> [244.528629] ? __pfx_kthread+0x10/0x10
<4> [244.528643] ret_from_fork_asm+0x1a/0x30
<4> [244.528654] </TASK>
<4> [244.528656] irq event stamp: 1327129
<4> [244.528658] hardirqs last enabled at (1327135): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.528662] hardirqs last disabled at (1327140): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.528665] softirqs last enabled at (1327112): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.528669] softirqs last disabled at (1327099): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.528672] ---[ end trace 0000000000000000 ]---
<7> [244.528613] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.528763] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [244.528823] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.528958] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.528971] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.529381] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.529653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.529798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.529944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.530085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.530224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.530362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.530502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.530650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.530788] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.530937] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.531083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.532214] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.542640] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.542907] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.543942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.544032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.544118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.544202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.544283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.544362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.544439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.544517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.544593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.544679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.544755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.544830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.544906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.544981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.545057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.545132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.545206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.545291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.545366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.545452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.545537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.545620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.545724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.545809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.545893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.545973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.546051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.546129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.546206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.546286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.546367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.546446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.546525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.546605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.546696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.546782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.546866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.546946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.547026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.547103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.547181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.547258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.547335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.547412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.547492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.547571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.547659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.547739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.547818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.547897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.547978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.548055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.548133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.548210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.548288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.548366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.548444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.548522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.548598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.548693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.548770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.548848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.548924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.549000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.549076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.549152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.549228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.549302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.549380] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.549506] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.549510] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38114, lrc_seqno=38114, guc_id=0, flags=0x73 in no process [-1]
<7> [244.549513] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.549581] ------------[ cut here ]------------
<4> [244.549582] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.549583] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:8/983
<4> [244.549674] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.549740]
<4> [244.549743] CPU: 2 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.549746] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.549747] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.549749] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.549754] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.549830] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.549832] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [244.549834] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.549836] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.549837] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.549838] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.549839] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.549841] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [244.549842] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.549844] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.549845] PKRU: 55555554
<4> [244.549846] Call Trace:
<4> [244.549847] <TASK>
<4> [244.549852] ? lock_acquire+0x60/0x300
<4> [244.549857] ? lock_release+0xd0/0x2b0
<4> [244.549862] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.549868] process_one_work+0x239/0x740
<4> [244.549874] worker_thread+0x200/0x3f0
<4> [244.549877] ? __pfx_worker_thread+0x10/0x10
<4> [244.549880] kthread+0x10d/0x150
<4> [244.549882] ? __pfx_kthread+0x10/0x10
<4> [244.549885] ret_from_fork+0x3bd/0x470
<4> [244.549887] ? __pfx_kthread+0x10/0x10
<4> [244.549890] ret_from_fork_asm+0x1a/0x30
<4> [244.549897] </TASK>
<4> [244.549898] irq event stamp: 1330347
<4> [244.549900] hardirqs last enabled at (1330353): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.549902] hardirqs last disabled at (1330358): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.549904] softirqs last enabled at (1329436): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.549906] softirqs last disabled at (1329393): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.549909] ---[ end trace 0000000000000000 ]---
<7> [244.550939] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.551098] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.552722] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [244.552816] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [244.552899] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.552981] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553061] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553136] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553210] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.553282] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553355] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553425] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.553511] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [244.553620] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [244.553744] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [244.553865] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [244.564203] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [244.648144] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.648735] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.649188] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.649607] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.650099] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.650505] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.650964] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.651361] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.651793] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.652173] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.652619] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.652777] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [244.652881] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [244.652983] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [244.698404] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38115, lrc_seqno=38115, guc_id=0, flags=0x73 in no process [-1]
<7> [244.698432] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.698982] ------------[ cut here ]------------
<4> [244.698991] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.698999] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#13: kworker/u64:2/121
<4> [244.699527] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.700010]
<4> [244.700025] CPU: 13 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.700044] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.700052] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.700062] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.700099] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.700598] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.700608] RSP: 0000:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.700622] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.700649] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.700658] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.700666] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.700673] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.700681] FS: 0000000000000000(0000) GS:ffff8888db303000(0000) knlGS:0000000000000000
<4> [244.700691] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.700700] CR2: 0000718d919de000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.700709] PKRU: 55555554
<4> [244.700716] Call Trace:
<4> [244.700723] <TASK>
<4> [244.700750] ? lock_acquire+0x60/0x300
<4> [244.700785] ? lock_release+0xd0/0x2b0
<4> [244.700817] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.700858] process_one_work+0x239/0x740
<4> [244.700904] worker_thread+0x200/0x3f0
<4> [244.700924] ? __pfx_worker_thread+0x10/0x10
<4> [244.700939] kthread+0x10d/0x150
<4> [244.700952] ? __pfx_kthread+0x10/0x10
<4> [244.700971] ret_from_fork+0x3bd/0x470
<4> [244.700985] ? __pfx_kthread+0x10/0x10
<4> [244.701001] ret_from_fork_asm+0x1a/0x30
<4> [244.701053] </TASK>
<4> [244.701060] irq event stamp: 679151
<4> [244.701067] hardirqs last enabled at (679157): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.701081] hardirqs last disabled at (679162): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.701091] softirqs last enabled at (678384): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.701108] softirqs last disabled at (678377): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.701123] ---[ end trace 0000000000000000 ]---
<6> [244.701283] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.701908] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.701972] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.702289] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.703032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.703561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.704142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.704624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.705144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.705596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.706115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.706600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.707173] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.707734] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.708254] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.710193] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.720644] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.720970] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.722151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.722260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.722365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.722469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.722564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.722661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.722754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.722842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.722929] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.723015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.723098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.723181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.723264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.723347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.723429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.723509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.723589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.723676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.723757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.723848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.723937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.724023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.724108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.724195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.724281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.724368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.724450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.724531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.724612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.724709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.724795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.724877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.724961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.725044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.725133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.725222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.725309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.725394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.725479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.725561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.725648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.725731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.725811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.725893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.725975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.726059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.726141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.726223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.726306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.726388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.726473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.726553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.726639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.726721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.726802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.726882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.726962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.727041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.727123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.727206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.727305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.727388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.727470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.727552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.727643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.727726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.727805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.727886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.727969] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.728095] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.728705] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38115, lrc_seqno=38115, guc_id=0, flags=0x73 in no process [-1]
<7> [244.728712] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.728844] ------------[ cut here ]------------
<4> [244.728847] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.728849] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [244.728941] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.729011]
<4> [244.729013] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.729017] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.729018] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.729019] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.729026] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.729112] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.729114] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.729117] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.729118] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.729120] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.729121] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.729122] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.729123] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [244.729125] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.729126] CR2: 0000718d911e3000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.729128] PKRU: 55555554
<4> [244.729129] Call Trace:
<4> [244.729130] <TASK>
<4> [244.729134] ? lock_acquire+0x60/0x300
<4> [244.729141] ? lock_release+0xd0/0x2b0
<7> [244.729077] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [244.729146] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.729152] process_one_work+0x239/0x740
<4> [244.729160] worker_thread+0x200/0x3f0
<4> [244.729163] ? __pfx_worker_thread+0x10/0x10
<4> [244.729165] kthread+0x10d/0x150
<4> [244.729167] ? __pfx_kthread+0x10/0x10
<4> [244.729170] ret_from_fork+0x3bd/0x470
<4> [244.729173] ? __pfx_kthread+0x10/0x10
<4> [244.729175] ret_from_fork_asm+0x1a/0x30
<4> [244.729183] </TASK>
<4> [244.729184] irq event stamp: 682465
<4> [244.729185] hardirqs last enabled at (682471): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.729188] hardirqs last disabled at (682476): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.729189] softirqs last enabled at (681586): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.729193] softirqs last disabled at (681545): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.729195] ---[ end trace 0000000000000000 ]---
<6> [244.729197] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [244.729207] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [244.729280] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.729287] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.729700] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.729919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.730024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.730130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.730232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.730334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.730430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.730522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.730616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.730719] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.730828] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.730928] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.731969] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.742658] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.742924] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.743931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.744023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.744110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.744196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.744280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.744363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.744445] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.744526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.744607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.744698] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.744780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.744860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.744940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.745021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.745101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.745181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.745261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.745342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.745421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.745513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.745603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.745698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.745783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.745869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.745954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.746038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.746122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.746204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.746287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.746373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.746459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.746543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.746628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.746728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.746822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.746912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.747002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.747089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.747175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.747258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.747340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.747422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.747503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.747585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.747675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.747759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.747841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.747923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.748004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.748087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.748171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.748254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.748337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.748419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.748500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.748583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.748673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.748754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.748834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.748917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.749000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.749082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.749163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.749245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.749330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.749415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.749497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.749578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.749670] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.749801] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.749805] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38115, lrc_seqno=38115, guc_id=0, flags=0x73 in no process [-1]
<7> [244.749807] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.749881] ------------[ cut here ]------------
<4> [244.749882] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.749883] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [244.749967] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.750034]
<4> [244.750036] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.750039] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.750040] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.750041] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.750048] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.750129] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.750131] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.750133] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.750135] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.750136] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.750137] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.750139] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.750140] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [244.750141] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.750143] CR2: 0000718d911e3000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.750144] PKRU: 55555554
<4> [244.750146] Call Trace:
<4> [244.750147] <TASK>
<4> [244.750151] ? lock_acquire+0x60/0x300
<4> [244.750156] ? lock_release+0xd0/0x2b0
<4> [244.750161] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.750167] process_one_work+0x239/0x740
<4> [244.750174] worker_thread+0x200/0x3f0
<4> [244.750177] ? __pfx_worker_thread+0x10/0x10
<4> [244.750180] kthread+0x10d/0x150
<4> [244.750182] ? __pfx_kthread+0x10/0x10
<4> [244.750185] ret_from_fork+0x3bd/0x470
<4> [244.750188] ? __pfx_kthread+0x10/0x10
<4> [244.750190] ret_from_fork_asm+0x1a/0x30
<4> [244.750198] </TASK>
<4> [244.750199] irq event stamp: 685627
<4> [244.750200] hardirqs last enabled at (685633): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.750203] hardirqs last disabled at (685638): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.750205] softirqs last enabled at (684352): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.750208] softirqs last disabled at (684347): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.750210] ---[ end trace 0000000000000000 ]---
<7> [244.751513] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.751646] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [244.757380] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38123, lrc_seqno=38123, guc_id=0, flags=0x73 in no process [-1]
<7> [244.757384] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.757458] ------------[ cut here ]------------
<4> [244.757459] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.757461] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [244.757550] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.757616]
<4> [244.757618] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.757621] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.757622] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.757624] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.757629] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.757722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.757724] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.757726] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.757728] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.757729] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.757730] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.757732] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.757733] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [244.757735] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.757737] CR2: 0000718d911e3000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [244.757739] PKRU: 55555554
<4> [244.757740] Call Trace:
<4> [244.757741] <TASK>
<4> [244.757745] ? lock_acquire+0x60/0x300
<4> [244.757751] ? lock_release+0xd0/0x2b0
<4> [244.757756] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.757762] process_one_work+0x239/0x740
<4> [244.757769] worker_thread+0x200/0x3f0
<4> [244.757772] ? __pfx_worker_thread+0x10/0x10
<4> [244.757775] kthread+0x10d/0x150
<4> [244.757777] ? __pfx_kthread+0x10/0x10
<4> [244.757780] ret_from_fork+0x3bd/0x470
<4> [244.757783] ? __pfx_kthread+0x10/0x10
<4> [244.757785] ret_from_fork_asm+0x1a/0x30
<4> [244.757794] </TASK>
<4> [244.757795] irq event stamp: 686651
<4> [244.757796] hardirqs last enabled at (686657): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.757798] hardirqs last disabled at (686662): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.757800] softirqs last enabled at (686490): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.757803] softirqs last disabled at (686483): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.757806] ---[ end trace 0000000000000000 ]---
<6> [244.757818] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.757896] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.757902] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.758180] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.758406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.758500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.758597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.758698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.758789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.758879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.758968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.759057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.759140] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.759241] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.759334] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.760369] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.770639] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.770900] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.771861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.771947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.772028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.772110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.772190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.772268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.772344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.772420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.772495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.772570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.772652] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.772728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.772804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.772880] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.772956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.773031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.773106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.773180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.773255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.773340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.773425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.773506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.773584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.773677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.773757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.773834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.773912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.773987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.774064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.774144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.774224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.774302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.774381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.774460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.774544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.774627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.774724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.774805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.774885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.774962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.775039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.775115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.775191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.775267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.775344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.775422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.775500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.775576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.775655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.775733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.775812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.775888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.775966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.776043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.776118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.776195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.776273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.776349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.776424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.776502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.776579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.776661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.776743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.776820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.776900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.776978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.777052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.777127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.777205] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.777328] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.777333] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38123, lrc_seqno=38123, guc_id=0, flags=0x73 in no process [-1]
<7> [244.777335] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.777402] ------------[ cut here ]------------
<4> [244.777403] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.777405] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:2/121
<4> [244.777483] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.777551]
<4> [244.777553] CPU: 2 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.777556] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.777558] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.777559] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.777565] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.777644] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.777646] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.777648] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.777650] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.777651] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.777653] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.777654] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.777656] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [244.777657] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.777659] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.777661] PKRU: 55555554
<4> [244.777662] Call Trace:
<4> [244.777663] <TASK>
<4> [244.777667] ? lock_acquire+0x60/0x300
<4> [244.777672] ? lock_release+0xd0/0x2b0
<4> [244.777677] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.777683] process_one_work+0x239/0x740
<4> [244.777690] worker_thread+0x200/0x3f0
<4> [244.777693] ? __pfx_worker_thread+0x10/0x10
<4> [244.777696] kthread+0x10d/0x150
<4> [244.777698] ? __pfx_kthread+0x10/0x10
<4> [244.777701] ret_from_fork+0x3bd/0x470
<4> [244.777704] ? __pfx_kthread+0x10/0x10
<4> [244.777707] ret_from_fork_asm+0x1a/0x30
<4> [244.777714] </TASK>
<4> [244.777716] irq event stamp: 689805
<4> [244.777717] hardirqs last enabled at (689811): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.777719] hardirqs last disabled at (689816): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.777721] softirqs last enabled at (688636): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.777724] softirqs last disabled at (688591): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.777727] ---[ end trace 0000000000000000 ]---
<6> [244.778646] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.778791] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.778803] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.779006] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.779117] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.779227] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.779446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.779546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.779651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.779742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.779832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.779920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.780008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.780097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.780180] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.780281] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.780376] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.781429] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.791649] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.791910] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.793395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.793491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.793578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.793671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.793755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.793836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.793916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.793994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.794072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.794149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.794225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.794302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.794377] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.794453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.794530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.794606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.794695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.794771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.794846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.794933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.795017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.795100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.795180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.795262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.795347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.795427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.795504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.795582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.795669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.795751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.795833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.795913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.795993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.796074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.796159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.796245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.796329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.796414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.796497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.796576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.796664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.796742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.796818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.796896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.796976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.797055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.797133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.797210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.797287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.797363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.797443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.797521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.797597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.797685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.797760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.797838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.797915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.797990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.798069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.798150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.798227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.798304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.798380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.798456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.798532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.798610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.798698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.798775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.798853] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.798981] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.798986] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38123, lrc_seqno=38123, guc_id=0, flags=0x73 in no process [-1]
<7> [244.798988] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.799056] ------------[ cut here ]------------
<4> [244.799058] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.799059] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:2/121
<4> [244.799138] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.799204]
<4> [244.799206] CPU: 10 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.799209] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.799211] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.799212] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.799219] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.799294] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.799296] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.799299] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.799300] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.799301] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.799302] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.799304] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.799305] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [244.799307] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.799308] CR2: 00007fe398454000 CR3: 00000001227a0000 CR4: 0000000000f52ef0
<4> [244.799309] PKRU: 55555554
<4> [244.799311] Call Trace:
<4> [244.799312] <TASK>
<4> [244.799316] ? lock_acquire+0x60/0x300
<4> [244.799321] ? lock_release+0xd0/0x2b0
<4> [244.799326] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.799332] process_one_work+0x239/0x740
<4> [244.799339] worker_thread+0x200/0x3f0
<4> [244.799342] ? __pfx_worker_thread+0x10/0x10
<4> [244.799345] kthread+0x10d/0x150
<4> [244.799347] ? __pfx_kthread+0x10/0x10
<4> [244.799350] ret_from_fork+0x3bd/0x470
<4> [244.799352] ? __pfx_kthread+0x10/0x10
<4> [244.799355] ret_from_fork_asm+0x1a/0x30
<4> [244.799362] </TASK>
<4> [244.799363] irq event stamp: 693025
<4> [244.799365] hardirqs last enabled at (693031): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.799367] hardirqs last disabled at (693036): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.799368] softirqs last enabled at (691890): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.799372] softirqs last disabled at (691883): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.799374] ---[ end trace 0000000000000000 ]---
<7> [244.800646] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.800766] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [244.801550] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [244.801662] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [244.801747] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.801824] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.801898] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.801970] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.802042] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [244.802112] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.802184] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.802252] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [244.802336] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [244.802445] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [244.802552] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [244.802658] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [244.814215] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [244.898106] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.898713] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.899179] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.899572] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.900038] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.900413] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.900828] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.901188] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.901548] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.901969] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.902398] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.902906] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [244.902974] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [244.903037] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.903099] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.903164] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [244.903228] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [244.903291] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [244.903353] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [244.903413] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [244.903473] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [244.903537] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [244.903641] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [244.903747] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [244.903848] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [244.948407] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38124, lrc_seqno=38124, guc_id=0, flags=0x73 in no process [-1]
<7> [244.948432] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.948952] ------------[ cut here ]------------
<4> [244.948959] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.948967] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [244.949435] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.949846]
<4> [244.949860] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.949877] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.949884] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.949893] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.949924] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.950357] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.950366] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [244.950379] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.950387] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.950393] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.950399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.950405] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.950412] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [244.950420] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.950427] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.950435] PKRU: 55555554
<4> [244.950441] Call Trace:
<4> [244.950447] <TASK>
<4> [244.950468] ? lock_acquire+0x60/0x300
<4> [244.950496] ? lock_release+0xd0/0x2b0
<4> [244.950520] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.950551] process_one_work+0x239/0x740
<4> [244.950586] worker_thread+0x200/0x3f0
<4> [244.950602] ? __pfx_worker_thread+0x10/0x10
<4> [244.950616] kthread+0x10d/0x150
<4> [244.950627] ? __pfx_kthread+0x10/0x10
<4> [244.950660] ret_from_fork+0x3bd/0x470
<4> [244.950672] ? __pfx_kthread+0x10/0x10
<4> [244.950686] ret_from_fork_asm+0x1a/0x30
<4> [244.950725] </TASK>
<4> [244.950731] irq event stamp: 694379
<4> [244.950737] hardirqs last enabled at (694385): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.950750] hardirqs last disabled at (694390): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.950758] softirqs last enabled at (693614): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.950775] softirqs last disabled at (693609): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.950788] ---[ end trace 0000000000000000 ]---
<6> [244.950798] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.951255] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.951309] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.951538] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.952420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.953050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.953689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.954161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.954601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.955084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.955514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.955997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.956428] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.956979] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.957539] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.959477] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.969642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.969943] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.971033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.971130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.971228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.971313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.971396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.971478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.971554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.971629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.971719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.971795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.971871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.971947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.972022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.972096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.972169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.972242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.972316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.972389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.972464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.972549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.972637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.972718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.972796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.972875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.972955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.973033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.973110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.973203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.973281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.973361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.973441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.973520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.973599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.973701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.973785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.973868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.973952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.974032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.974116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.974193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.974270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.974347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.974422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.974500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.974578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.974660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.974737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.974813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.974889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.974970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.975054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.975129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.975207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.975284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.975361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.975441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.975521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.975596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.975680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.975760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.975837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.975914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.975991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.976067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.976143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.976219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.976295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.976370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.976449] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.976571] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.976576] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38124, lrc_seqno=38124, guc_id=0, flags=0x73 in no process [-1]
<7> [244.976578] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.976652] ------------[ cut here ]------------
<4> [244.976653] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.976655] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [244.976733] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.976797]
<4> [244.976800] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.976803] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.976804] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.976806] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.976811] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.976887] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.976888] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [244.976890] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.976892] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.976893] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.976894] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.976895] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.976897] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [244.976899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.976900] CR2: 00007fe39844b000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.976901] PKRU: 55555554
<4> [244.976903] Call Trace:
<4> [244.976904] <TASK>
<4> [244.976908] ? lock_acquire+0x60/0x300
<4> [244.976913] ? lock_release+0xd0/0x2b0
<4> [244.976918] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.976924] process_one_work+0x239/0x740
<4> [244.976930] worker_thread+0x200/0x3f0
<4> [244.976933] ? __pfx_worker_thread+0x10/0x10
<4> [244.976936] kthread+0x10d/0x150
<4> [244.976938] ? __pfx_kthread+0x10/0x10
<4> [244.976941] ret_from_fork+0x3bd/0x470
<4> [244.976943] ? __pfx_kthread+0x10/0x10
<4> [244.976946] ret_from_fork_asm+0x1a/0x30
<4> [244.976954] </TASK>
<4> [244.976955] irq event stamp: 216293
<4> [244.976956] hardirqs last enabled at (216299): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.976958] hardirqs last disabled at (216304): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.976960] softirqs last enabled at (215290): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.976962] softirqs last disabled at (215283): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.976965] ---[ end trace 0000000000000000 ]---
<6> [244.976967] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [244.977044] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [244.977050] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [244.977465] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [244.977679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [244.977773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [244.977869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [244.977958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [244.978046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [244.978132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [244.978218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [244.978305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [244.978387] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [244.978487] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [244.978581] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [244.979619] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [244.989639] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [244.989893] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [244.990975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [244.991058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [244.991138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [244.991216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [244.991294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [244.991375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [244.991453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [244.991531] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [244.991607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [244.991693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [244.991769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [244.991845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [244.991920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [244.991995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [244.992071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [244.992146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [244.992221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [244.992297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [244.992373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [244.992458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [244.992542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [244.992625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [244.992717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [244.992797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [244.992875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [244.992952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [244.993028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [244.993104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [244.993179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [244.993258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [244.993337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [244.993414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [244.993492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [244.993570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [244.993660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [244.993743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [244.993826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [244.993904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [244.993982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [244.994058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [244.994133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [244.994209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [244.994283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [244.994358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [244.994435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [244.994511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [244.994591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [244.994675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [244.994750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [244.994826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [244.994907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [244.994982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [244.995058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [244.995133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [244.995207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [244.995286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [244.995362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [244.995438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [244.995512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [244.995590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [244.995674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [244.995754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [244.995832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [244.995911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [244.995990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [244.996069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [244.996144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [244.996218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [244.996297] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [244.996415] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [244.996419] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38124, lrc_seqno=38124, guc_id=0, flags=0x73 in no process [-1]
<7> [244.996422] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [244.996488] ------------[ cut here ]------------
<4> [244.996490] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [244.996491] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:4/203
<4> [244.996569] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [244.996639]
<4> [244.996641] CPU: 4 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [244.996644] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [244.996646] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [244.996647] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [244.996653] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [244.996730] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [244.996732] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [244.996734] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [244.996736] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [244.996737] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [244.996738] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [244.996740] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [244.996741] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [244.996742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [244.996744] CR2: 00007fe39844b000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [244.996745] PKRU: 55555554
<4> [244.996746] Call Trace:
<4> [244.996748] <TASK>
<4> [244.996752] ? lock_acquire+0x60/0x300
<4> [244.996757] ? lock_release+0xd0/0x2b0
<4> [244.996761] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [244.996768] process_one_work+0x239/0x740
<4> [244.996774] worker_thread+0x200/0x3f0
<4> [244.996777] ? __pfx_worker_thread+0x10/0x10
<4> [244.996780] kthread+0x10d/0x150
<4> [244.996782] ? __pfx_kthread+0x10/0x10
<4> [244.996785] ret_from_fork+0x3bd/0x470
<4> [244.996787] ? __pfx_kthread+0x10/0x10
<4> [244.996789] ret_from_fork_asm+0x1a/0x30
<4> [244.996797] </TASK>
<4> [244.996798] irq event stamp: 219529
<4> [244.996799] hardirqs last enabled at (219535): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [244.996801] hardirqs last disabled at (219540): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [244.996803] softirqs last enabled at (218356): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.996805] softirqs last disabled at (218349): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [244.996808] ---[ end trace 0000000000000000 ]---
<7> [244.998058] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [244.998172] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [245.004418] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38132, lrc_seqno=38132, guc_id=0, flags=0x73 in no process [-1]
<7> [245.004423] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.004509] ------------[ cut here ]------------
<4> [245.004511] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.004512] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:4/203
<4> [245.004603] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.004675]
<4> [245.004678] CPU: 2 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.004681] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.004682] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.004683] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.004689] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.004775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.004777] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [245.004780] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.004781] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.004782] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.004783] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.004784] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.004786] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [245.004787] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.004789] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.004790] PKRU: 55555554
<4> [245.004791] Call Trace:
<4> [245.004793] <TASK>
<4> [245.004797] ? lock_acquire+0x60/0x300
<4> [245.004802] ? lock_release+0xd0/0x2b0
<4> [245.004807] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.004813] process_one_work+0x239/0x740
<4> [245.004820] worker_thread+0x200/0x3f0
<4> [245.004823] ? __pfx_worker_thread+0x10/0x10
<4> [245.004825] kthread+0x10d/0x150
<4> [245.004828] ? __pfx_kthread+0x10/0x10
<4> [245.004831] ret_from_fork+0x3bd/0x470
<4> [245.004833] ? __pfx_kthread+0x10/0x10
<4> [245.004835] ret_from_fork_asm+0x1a/0x30
<4> [245.004843] </TASK>
<4> [245.004844] irq event stamp: 220567
<4> [245.004845] hardirqs last enabled at (220573): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.004848] hardirqs last disabled at (220578): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.004849] softirqs last enabled at (220048): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.004852] softirqs last disabled at (220043): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.004855] ---[ end trace 0000000000000000 ]---
<6> [245.004857] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.004939] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.004946] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.005354] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.005578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.005695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.005799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.005894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.005983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.006071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.006159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.006247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.006330] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.006431] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.006525] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.007565] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.017642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.017900] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.018931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.019017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.019096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.019177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.019255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.019332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.019409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.019485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.019559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.019688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.019818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.019938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.020055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.020172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.020295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.020417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.020506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.020586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.020695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.020783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.020869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.020951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.021031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.021110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.021189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.021266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.021343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.021419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.021496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.021576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.021668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.021749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.021828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.021907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.021991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.022073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.022157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.022237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.022316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.022393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.022470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.022546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.022621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.022713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.022791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.022868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.022945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.023020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.023094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.023171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.023248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.023323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.023398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.023477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.023552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.023636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.023712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.023785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.023859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.023936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.024013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.024092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.024169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.024245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.024322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.024397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.024476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.024553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.024638] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.024762] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.024767] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38132, lrc_seqno=38132, guc_id=0, flags=0x73 in no process [-1]
<7> [245.024769] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.024837] ------------[ cut here ]------------
<4> [245.024838] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.024840] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:8/983
<4> [245.024918] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.024983]
<4> [245.024985] CPU: 11 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.024988] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.024990] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.024991] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.024997] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.025073] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.025074] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.025077] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.025078] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.025079] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.025080] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.025082] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.025083] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [245.025085] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.025086] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [245.025088] PKRU: 55555554
<4> [245.025089] Call Trace:
<4> [245.025090] <TASK>
<4> [245.025094] ? lock_acquire+0x60/0x300
<4> [245.025099] ? lock_release+0xd0/0x2b0
<4> [245.025104] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.025110] process_one_work+0x239/0x740
<4> [245.025116] worker_thread+0x200/0x3f0
<4> [245.025119] ? __pfx_worker_thread+0x10/0x10
<4> [245.025122] kthread+0x10d/0x150
<4> [245.025124] ? __pfx_kthread+0x10/0x10
<4> [245.025127] ret_from_fork+0x3bd/0x470
<4> [245.025129] ? __pfx_kthread+0x10/0x10
<4> [245.025132] ret_from_fork_asm+0x1a/0x30
<4> [245.025139] </TASK>
<4> [245.025140] irq event stamp: 1336173
<4> [245.025141] hardirqs last enabled at (1336179): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.025144] hardirqs last disabled at (1336184): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.025145] softirqs last enabled at (1334972): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.025148] softirqs last disabled at (1334963): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.025150] ---[ end trace 0000000000000000 ]---
<6> [245.025152] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.025227] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.025234] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.025619] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.025845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.025935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.026028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.026115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.026202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.026288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.026374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.026462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.026545] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.026654] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.026749] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.027784] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.037642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.037906] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.038870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.038959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.039043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.039126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.039207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.039287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.039368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.039447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.039525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.039600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.039689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.039767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.039844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.039921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.039997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.040072] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.040147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.040222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.040297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.040383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.040468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.040550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.040628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.040725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.040805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.040883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.040960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.041037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.041114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.041193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.041271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.041349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.041426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.041505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.041588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.041682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.041767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.041848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.041928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.042004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.042082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.042158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.042234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.042312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.042392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.042470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.042548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.042626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.042719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.042798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.042880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.043033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.043158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.043278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.043395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.043521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.043653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.043779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.043911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.044043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.044136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.044222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.044304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.044384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.044466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.044547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.044626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.044725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.044810] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.044937] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.044941] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38132, lrc_seqno=38132, guc_id=0, flags=0x73 in no process [-1]
<7> [245.044944] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.045013] ------------[ cut here ]------------
<4> [245.045014] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.045015] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:8/983
<4> [245.045094] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.045157]
<4> [245.045159] CPU: 11 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.045162] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.045163] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.045165] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.045170] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.045248] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.045249] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.045252] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.045253] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.045254] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.045256] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.045257] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.045258] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [245.045260] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.045261] CR2: 0000718d91c1e000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [245.045263] PKRU: 55555554
<4> [245.045264] Call Trace:
<4> [245.045265] <TASK>
<4> [245.045269] ? lock_acquire+0x60/0x300
<4> [245.045274] ? lock_release+0xd0/0x2b0
<4> [245.045279] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.045285] process_one_work+0x239/0x740
<4> [245.045291] worker_thread+0x200/0x3f0
<4> [245.045294] ? __pfx_worker_thread+0x10/0x10
<4> [245.045297] kthread+0x10d/0x150
<4> [245.045299] ? __pfx_kthread+0x10/0x10
<4> [245.045302] ret_from_fork+0x3bd/0x470
<4> [245.045304] ? __pfx_kthread+0x10/0x10
<4> [245.045307] ret_from_fork_asm+0x1a/0x30
<4> [245.045314] </TASK>
<4> [245.045315] irq event stamp: 1339325
<4> [245.045316] hardirqs last enabled at (1339331): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.045319] hardirqs last disabled at (1339336): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.045320] softirqs last enabled at (1338468): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.045323] softirqs last disabled at (1338461): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.045325] ---[ end trace 0000000000000000 ]---
<7> [245.046654] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.046772] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.047533] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [245.047644] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [245.047735] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.047813] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.047886] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.047959] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048031] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.048100] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048169] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048237] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048321] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [245.048510] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [245.048594] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [245.048681] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.048756] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048828] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048900] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.048972] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.049043] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.049115] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.049185] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.049259] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [245.049364] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [245.049466] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [245.049568] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [245.064203] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [245.148042] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.148543] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.149094] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.149493] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.149975] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.150351] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.150753] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.151112] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.151470] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.151872] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.152310] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.152691] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.152761] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.152827] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.152889] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.152950] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.153011] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.153072] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.153138] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.153203] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.153266] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.153332] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.153428] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [245.153522] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [245.153610] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [245.197815] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38133, lrc_seqno=38133, guc_id=0, flags=0x73 in no process [-1]
<7> [245.197841] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.198272] ------------[ cut here ]------------
<4> [245.198279] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.198287] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:4/203
<4> [245.198795] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.199156]
<4> [245.199169] CPU: 2 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.199186] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.199194] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.199202] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.199234] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.199688] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.199698] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [245.199712] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.199720] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.199728] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.199736] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.199743] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.199751] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [245.199760] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.199768] CR2: 00007fe398458000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.199776] PKRU: 55555554
<4> [245.199783] Call Trace:
<4> [245.199790] <TASK>
<4> [245.199813] ? lock_acquire+0x60/0x300
<4> [245.199841] ? lock_release+0xd0/0x2b0
<4> [245.199866] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.199898] process_one_work+0x239/0x740
<4> [245.199934] worker_thread+0x200/0x3f0
<4> [245.199951] ? __pfx_worker_thread+0x10/0x10
<4> [245.199965] kthread+0x10d/0x150
<4> [245.199978] ? __pfx_kthread+0x10/0x10
<4> [245.199994] ret_from_fork+0x3bd/0x470
<4> [245.200007] ? __pfx_kthread+0x10/0x10
<4> [245.200022] ret_from_fork_asm+0x1a/0x30
<4> [245.200063] </TASK>
<4> [245.200070] irq event stamp: 222411
<4> [245.200077] hardirqs last enabled at (222417): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.200090] hardirqs last disabled at (222422): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.200099] softirqs last enabled at (221666): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.200117] softirqs last disabled at (221661): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.200131] ---[ end trace 0000000000000000 ]---
<6> [245.200142] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.200561] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.200596] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.201095] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.201897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.202654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.203274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.203790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.204241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.204767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.205195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.205696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.206125] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.206631] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.207210] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.209066] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.220061] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.220345] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.221401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.221489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.221570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.221659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.221740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.221820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.221903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.221985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.222067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.222148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.222227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.222304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.222381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.222459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.222534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.222609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.222696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.222772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.222846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.222931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.223014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.223094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.223172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.223252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.223332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.223412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.223489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.223566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.223650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.223730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.223809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.223887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.223964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.224042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.224125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.224207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.224291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.224369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.224448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.224523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.224601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.224687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.224763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.224840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.224923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.225012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.225091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.225169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.225244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.225322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.225402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.225477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.225555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.225709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.225836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.225917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.225998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.226076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.226153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.226233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.226311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.226388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.226465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.226540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.226617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.226710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.226787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.226867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.226947] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.227079] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.227084] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38133, lrc_seqno=38133, guc_id=0, flags=0x73 in no process [-1]
<7> [245.227086] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.227155] ------------[ cut here ]------------
<4> [245.227156] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.227157] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:8/983
<4> [245.227236] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.227301]
<4> [245.227304] CPU: 11 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.227307] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.227308] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.227309] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.227315] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.227391] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.227393] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.227395] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.227397] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.227398] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.227399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.227400] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.227402] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [245.227403] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.227405] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.227406] PKRU: 55555554
<4> [245.227407] Call Trace:
<4> [245.227408] <TASK>
<4> [245.227412] ? lock_acquire+0x60/0x300
<4> [245.227418] ? lock_release+0xd0/0x2b0
<4> [245.227422] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.227428] process_one_work+0x239/0x740
<4> [245.227435] worker_thread+0x200/0x3f0
<4> [245.227438] ? __pfx_worker_thread+0x10/0x10
<4> [245.227441] kthread+0x10d/0x150
<4> [245.227443] ? __pfx_kthread+0x10/0x10
<4> [245.227446] ret_from_fork+0x3bd/0x470
<4> [245.227449] ? __pfx_kthread+0x10/0x10
<4> [245.227451] ret_from_fork_asm+0x1a/0x30
<4> [245.227459] </TASK>
<4> [245.227460] irq event stamp: 1342495
<4> [245.227461] hardirqs last enabled at (1342501): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.227464] hardirqs last disabled at (1342506): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.227465] softirqs last enabled at (1341626): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.227468] softirqs last disabled at (1341619): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.227471] ---[ end trace 0000000000000000 ]---
<6> [245.227472] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.227950] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.227967] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.228913] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.229027] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.229139] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.229353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.229444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.229539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.229630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.229749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.229838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.229924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.230010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.230091] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.230191] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.230284] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.231347] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.241642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.241924] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.242961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.243046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.243127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.243207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.243286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.243365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.243443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.243520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.243599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.243687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.243763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.243840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.243916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.243993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.244069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.244146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.244222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.244298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.244373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.244459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.244543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.244625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.244720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.244801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.244880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.244960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.245041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.245119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.245196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.245276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.245356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.245435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.245515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.245594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.245687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.245771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.245855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.245936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.246015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.246092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.246170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.246246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.246322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.246400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.246478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.246555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.246636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.246714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.246789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.246867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.246947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.247024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.247101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.247178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.247254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.247332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.247408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.247483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.247558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.247639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.247716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.247793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.247870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.247945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.248022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.248099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.248174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.248267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.248348] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.248471] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.248476] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38133, lrc_seqno=38133, guc_id=0, flags=0x73 in no process [-1]
<7> [245.248479] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.248547] ------------[ cut here ]------------
<4> [245.248549] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.248550] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [245.248629] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.248699]
<4> [245.248701] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.248704] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.248706] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.248707] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.248713] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.248788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.248790] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.248792] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.248794] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.248795] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.248797] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.248798] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.248799] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [245.248801] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.248802] CR2: 00007fe398454000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.248804] PKRU: 55555554
<4> [245.248805] Call Trace:
<4> [245.248806] <TASK>
<4> [245.248810] ? lock_acquire+0x60/0x300
<4> [245.248815] ? lock_release+0xd0/0x2b0
<4> [245.248820] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.248826] process_one_work+0x239/0x740
<4> [245.248833] worker_thread+0x200/0x3f0
<4> [245.248836] ? __pfx_worker_thread+0x10/0x10
<4> [245.248839] kthread+0x10d/0x150
<4> [245.248841] ? __pfx_kthread+0x10/0x10
<4> [245.248844] ret_from_fork+0x3bd/0x470
<4> [245.248846] ? __pfx_kthread+0x10/0x10
<4> [245.248849] ret_from_fork_asm+0x1a/0x30
<4> [245.248857] </TASK>
<4> [245.248858] irq event stamp: 1345721
<4> [245.248859] hardirqs last enabled at (1345727): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.248861] hardirqs last disabled at (1345732): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.248863] softirqs last enabled at (1344628): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.248866] softirqs last disabled at (1344621): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.248868] ---[ end trace 0000000000000000 ]---
<7> [245.250196] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.250312] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [245.256068] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38141, lrc_seqno=38141, guc_id=0, flags=0x73 in no process [-1]
<7> [245.256072] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.256149] ------------[ cut here ]------------
<4> [245.256150] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.256152] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [245.256235] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.256300]
<4> [245.256302] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.256305] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.256306] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.256308] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.256313] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.256391] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.256393] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.256396] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.256397] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.256398] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.256399] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.256401] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.256402] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [245.256403] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.256405] CR2: 00007fe398454000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.256406] PKRU: 55555554
<4> [245.256407] Call Trace:
<4> [245.256409] <TASK>
<4> [245.256412] ? lock_acquire+0x60/0x300
<4> [245.256418] ? lock_release+0xd0/0x2b0
<4> [245.256422] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.256428] process_one_work+0x239/0x740
<4> [245.256435] worker_thread+0x200/0x3f0
<4> [245.256438] ? __pfx_worker_thread+0x10/0x10
<4> [245.256441] kthread+0x10d/0x150
<4> [245.256443] ? __pfx_kthread+0x10/0x10
<4> [245.256446] ret_from_fork+0x3bd/0x470
<4> [245.256448] ? __pfx_kthread+0x10/0x10
<4> [245.256451] ret_from_fork_asm+0x1a/0x30
<4> [245.256459] </TASK>
<4> [245.256460] irq event stamp: 1346737
<4> [245.256461] hardirqs last enabled at (1346743): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.256463] hardirqs last disabled at (1346748): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.256465] softirqs last enabled at (1344628): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.256468] softirqs last disabled at (1344621): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.256470] ---[ end trace 0000000000000000 ]---
<6> [245.256472] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.256549] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.256556] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.256861] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.257073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.257171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.257270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.257364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.257452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.257539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.257626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.257750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.257836] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.257938] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.258031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.259067] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.269642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.269902] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.270901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.270989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.271071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.271152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.271234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.271314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.271393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.271475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.271555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.271640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.271718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.271794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.271871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.271947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.272023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.272099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.272175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.272289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.272381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.272474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.272559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.272649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.272730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.272814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.272896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.272975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.273052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.273128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.273204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.273283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.273362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.273440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.273518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.273596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.273691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.273776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.273859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.273939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.274018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.274094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.274170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.274246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.274325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.274404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.274483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.274561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.274645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.274722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.274798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.274876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.274954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.275030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.275107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.275182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.275257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.275332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.275409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.275484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.275559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.275642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.275719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.275796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.275872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.275947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.276023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.276100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.276175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.276249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.276328] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.276456] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.276460] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38141, lrc_seqno=38141, guc_id=0, flags=0x73 in no process [-1]
<7> [245.276463] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.277390] ------------[ cut here ]------------
<4> [245.277391] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.277393] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#11: kworker/u64:2/121
<4> [245.277484] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.277556]
<4> [245.277558] CPU: 11 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.277561] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.277562] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.277564] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.277570] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.277651] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.277653] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [245.277655] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.277657] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.277658] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.277660] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.277661] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.277662] FS: 0000000000000000(0000) GS:ffff8888db203000(0000) knlGS:0000000000000000
<4> [245.277664] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.277665] CR2: 0000718d91c1e000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.277667] PKRU: 55555554
<4> [245.277668] Call Trace:
<4> [245.277669] <TASK>
<4> [245.277673] ? lock_acquire+0x60/0x300
<4> [245.277679] ? lock_release+0xd0/0x2b0
<4> [245.277684] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.277690] process_one_work+0x239/0x740
<4> [245.277697] worker_thread+0x200/0x3f0
<4> [245.277701] ? __pfx_worker_thread+0x10/0x10
<4> [245.277703] kthread+0x10d/0x150
<4> [245.277706] ? __pfx_kthread+0x10/0x10
<4> [245.277709] ret_from_fork+0x3bd/0x470
<4> [245.277711] ? __pfx_kthread+0x10/0x10
<4> [245.277714] ret_from_fork_asm+0x1a/0x30
<4> [245.277722] </TASK>
<4> [245.277723] irq event stamp: 698719
<4> [245.277724] hardirqs last enabled at (698725): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.277726] hardirqs last disabled at (698730): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.277728] softirqs last enabled at (697620): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.277731] softirqs last disabled at (697613): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.277734] ---[ end trace 0000000000000000 ]---
<6> [245.277736] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.277813] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.277820] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.278041] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.278153] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.278268] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.278484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.278580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.278690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.278781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.278871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.278958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.279046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.279134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.279219] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.279318] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.279411] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.280445] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.290642] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.290901] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.291897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.291983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.292064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.292144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.292222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.292298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.292374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.292450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.292524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.292599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.292685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.292762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.292837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.292913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.292990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.293064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.293139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.293214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.293288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.293373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.293456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.293537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.293618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.293712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.293793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.293872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.293950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.294029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.294107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.294186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.294267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.294347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.294426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.294505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.294590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.294685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.294770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.294850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.294931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.295008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.295086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.295163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.295239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.295317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.295395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.295472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.295550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.295665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.295763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.295842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.295923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.296000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.296078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.296154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.296230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.296307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.296384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.296459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.296535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.296614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.296713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.296792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.296869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.296947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.297025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.297103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.297178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.297254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.297332] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.297461] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.297465] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38141, lrc_seqno=38141, guc_id=0, flags=0x73 in no process [-1]
<7> [245.297467] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.298405] ------------[ cut here ]------------
<4> [245.298406] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.298408] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#10: kworker/u64:8/983
<4> [245.298492] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.298559]
<4> [245.298561] CPU: 10 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.298564] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.298566] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.298567] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.298573] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.298657] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.298659] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.298662] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.298664] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.298665] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.298666] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.298667] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.298669] FS: 0000000000000000(0000) GS:ffff8888db183000(0000) knlGS:0000000000000000
<4> [245.298670] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.298672] CR2: 00007fe398454000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.298673] PKRU: 55555554
<4> [245.298674] Call Trace:
<4> [245.298675] <TASK>
<4> [245.298679] ? lock_acquire+0x60/0x300
<4> [245.298685] ? lock_release+0xd0/0x2b0
<4> [245.298690] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.298696] process_one_work+0x239/0x740
<4> [245.298703] worker_thread+0x200/0x3f0
<4> [245.298706] ? __pfx_worker_thread+0x10/0x10
<4> [245.298709] kthread+0x10d/0x150
<4> [245.298711] ? __pfx_kthread+0x10/0x10
<4> [245.298714] ret_from_fork+0x3bd/0x470
<4> [245.298717] ? __pfx_kthread+0x10/0x10
<4> [245.298719] ret_from_fork_asm+0x1a/0x30
<4> [245.298728] </TASK>
<4> [245.298729] irq event stamp: 1350325
<4> [245.298730] hardirqs last enabled at (1350331): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.298732] hardirqs last disabled at (1350336): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.298734] softirqs last enabled at (1349416): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.298737] softirqs last disabled at (1349391): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.298740] ---[ end trace 0000000000000000 ]---
<7> [245.298778] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.298889] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.300465] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [245.300566] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [245.300668] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.300749] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.300826] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.300900] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.300973] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.301043] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.301113] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.301183] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.301267] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [245.301375] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [245.301481] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [245.301585] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [245.314235] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [245.398279] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.398899] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.399363] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.399826] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.400233] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.400631] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.401091] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.401480] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.401911] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.402293] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.402765] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.402933] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [245.403037] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [245.403139] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [245.448428] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38142, lrc_seqno=38142, guc_id=0, flags=0x73 in no process [-1]
<7> [245.448455] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.448999] ------------[ cut here ]------------
<4> [245.449007] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.449016] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#12: kworker/u64:8/983
<4> [245.449542] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.450021]
<4> [245.450038] CPU: 12 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.450057] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.450065] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.450075] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.450111] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.450608] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.450619] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.450652] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.450662] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.450670] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.450678] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.450685] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.450694] FS: 0000000000000000(0000) GS:ffff8888db283000(0000) knlGS:0000000000000000
<4> [245.450703] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.450711] CR2: 00007fe398457000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.450720] PKRU: 55555554
<4> [245.450727] Call Trace:
<4> [245.450735] <TASK>
<4> [245.450762] ? lock_acquire+0x60/0x300
<4> [245.450798] ? lock_release+0xd0/0x2b0
<4> [245.450832] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.450873] process_one_work+0x239/0x740
<4> [245.450918] worker_thread+0x200/0x3f0
<4> [245.450937] ? __pfx_worker_thread+0x10/0x10
<4> [245.450954] kthread+0x10d/0x150
<4> [245.450968] ? __pfx_kthread+0x10/0x10
<4> [245.450987] ret_from_fork+0x3bd/0x470
<4> [245.451000] ? __pfx_kthread+0x10/0x10
<4> [245.451017] ret_from_fork_asm+0x1a/0x30
<4> [245.451069] </TASK>
<4> [245.451076] irq event stamp: 1351735
<4> [245.451084] hardirqs last enabled at (1351741): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.451099] hardirqs last disabled at (1351746): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.451110] softirqs last enabled at (1350968): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.451128] softirqs last disabled at (1350961): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.451143] ---[ end trace 0000000000000000 ]---
<6> [245.451232] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.451809] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.451852] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.452214] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.452876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.453475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.454108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.454569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.455073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.455521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.455999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.456438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.456902] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.457409] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.457933] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.459770] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.470984] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.471320] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.472419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.472527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.472625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.472740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.472828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.472914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.472999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.473082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.473164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.473248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.473331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.473413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.473494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.473575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.473664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.473748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.473831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.473913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.473995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.474088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.474179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.474267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.474352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.474443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.474532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.474617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.474711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.474794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.474877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.474962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.475047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.475129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.475211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.475294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.475382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.475475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.475568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.475660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.475745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.475828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.475910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.475991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.476071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.476152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.476236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.476319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.476401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.476482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.476562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.476648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.476733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.476814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.476897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.476977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.477058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.477139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.477221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.477299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.477379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.477462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.477543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.477626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.477723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.477809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.477894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.477977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.478058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.478140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.478224] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.478369] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.478374] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38142, lrc_seqno=38142, guc_id=0, flags=0x73 in no process [-1]
<7> [245.478377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.478449] ------------[ cut here ]------------
<4> [245.478455] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.478457] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:8/983
<4> [245.478540] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.478611]
<4> [245.478614] CPU: 4 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.478618] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.478619] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.478621] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.478628] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.478722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.478724] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.478727] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.478729] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.478730] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.478732] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.478733] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.478735] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [245.478736] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.478738] CR2: 00007fe398459000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.478739] PKRU: 55555554
<4> [245.478741] Call Trace:
<4> [245.478742] <TASK>
<4> [245.478746] ? lock_acquire+0x60/0x300
<4> [245.478754] ? lock_release+0xd0/0x2b0
<4> [245.478759] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.478765] process_one_work+0x239/0x740
<4> [245.478774] worker_thread+0x200/0x3f0
<4> [245.478777] ? __pfx_worker_thread+0x10/0x10
<4> [245.478780] kthread+0x10d/0x150
<4> [245.478783] ? __pfx_kthread+0x10/0x10
<4> [245.478786] ret_from_fork+0x3bd/0x470
<4> [245.478789] ? __pfx_kthread+0x10/0x10
<4> [245.478792] ret_from_fork_asm+0x1a/0x30
<4> [245.478800] </TASK>
<4> [245.478801] irq event stamp: 1354911
<4> [245.478803] hardirqs last enabled at (1354917): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.478805] hardirqs last disabled at (1354922): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.478807] softirqs last enabled at (1354020): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.478811] softirqs last disabled at (1354009): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.478814] ---[ end trace 0000000000000000 ]---
<6> [245.479961] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.480045] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.480078] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.480134] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.480265] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.480385] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.480602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.480714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.480815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.480908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.480997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.481087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.481178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.481270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.481358] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.481464] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.481563] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.482608] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.492640] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.492902] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.494012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.494103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.494181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.494258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.494332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.494408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.494482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.494556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.494629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.494722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.494803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.494885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.494967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.495051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.495133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.495214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.495295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.495375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.495455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.495546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.495639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.495725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.495809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.495895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.495980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.496064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.496146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.496228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.496309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.496393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.496478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.496561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.496647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.496730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.496818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.496906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.496995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.497080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.497164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.497245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.497332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.497418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.497503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.497589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.497683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.497766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.497848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.497933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.498015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.498098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.498181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.498270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.498353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.498435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.498516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.498599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.498690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.498773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.498858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.498946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.499029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.499113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.499197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.499280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.499363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.499445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.499526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.499609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.499703] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.499829] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.499833] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38142, lrc_seqno=38142, guc_id=0, flags=0x73 in no process [-1]
<7> [245.499836] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.499908] ------------[ cut here ]------------
<4> [245.499909] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.499910] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:8/983
<4> [245.499993] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.500059]
<4> [245.500061] CPU: 4 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.500064] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.500065] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.500067] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.500073] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.500153] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.500154] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.500157] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.500158] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.500159] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.500161] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.500162] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.500163] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [245.500165] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.500166] CR2: 00007fe398459000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.500167] PKRU: 55555554
<4> [245.500169] Call Trace:
<4> [245.500170] <TASK>
<4> [245.500174] ? lock_acquire+0x60/0x300
<4> [245.500179] ? lock_release+0xd0/0x2b0
<4> [245.500184] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.500190] process_one_work+0x239/0x740
<4> [245.500197] worker_thread+0x200/0x3f0
<4> [245.500200] ? __pfx_worker_thread+0x10/0x10
<4> [245.500203] kthread+0x10d/0x150
<4> [245.500206] ? __pfx_kthread+0x10/0x10
<4> [245.500208] ret_from_fork+0x3bd/0x470
<4> [245.500211] ? __pfx_kthread+0x10/0x10
<4> [245.500214] ret_from_fork_asm+0x1a/0x30
<4> [245.500221] </TASK>
<4> [245.500222] irq event stamp: 1358081
<4> [245.500224] hardirqs last enabled at (1358087): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.500226] hardirqs last disabled at (1358092): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.500228] softirqs last enabled at (1357230): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.500231] softirqs last disabled at (1357223): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.500233] ---[ end trace 0000000000000000 ]---
<7> [245.501544] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.501677] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [245.507839] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38150, lrc_seqno=38150, guc_id=0, flags=0x73 in no process [-1]
<7> [245.507843] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.507919] ------------[ cut here ]------------
<4> [245.507920] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.507922] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:8/983
<4> [245.508004] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.508070]
<4> [245.508072] CPU: 4 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.508076] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.508077] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.508078] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.508084] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.508162] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.508164] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.508166] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.508167] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.508169] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.508170] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.508171] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.508172] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [245.508174] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.508175] CR2: 00007fe398459000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.508177] PKRU: 55555554
<4> [245.508178] Call Trace:
<4> [245.508179] <TASK>
<4> [245.508183] ? lock_acquire+0x60/0x300
<4> [245.508189] ? lock_release+0xd0/0x2b0
<4> [245.508193] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.508199] process_one_work+0x239/0x740
<4> [245.508206] worker_thread+0x200/0x3f0
<4> [245.508209] ? __pfx_worker_thread+0x10/0x10
<4> [245.508212] kthread+0x10d/0x150
<4> [245.508214] ? __pfx_kthread+0x10/0x10
<4> [245.508217] ret_from_fork+0x3bd/0x470
<4> [245.508220] ? __pfx_kthread+0x10/0x10
<4> [245.508222] ret_from_fork_asm+0x1a/0x30
<4> [245.508230] </TASK>
<4> [245.508232] irq event stamp: 1359507
<4> [245.508233] hardirqs last enabled at (1359513): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.508235] hardirqs last disabled at (1359518): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.508236] softirqs last enabled at (1358654): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.508239] softirqs last disabled at (1358649): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.508242] ---[ end trace 0000000000000000 ]---
<6> [245.508244] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.508324] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.508331] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.508609] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.508836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.508932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.509029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.509120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.509209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.509296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.509383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.509473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.509558] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.509664] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.509758] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.510791] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.520640] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.520898] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.521920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.522005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.522087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.522168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.522248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.522326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.522403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.522479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.522554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.522629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.522719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.522794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.522870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.522945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.523018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.523094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.523167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.523240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.523315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.523401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.523489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.523570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.523657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.523738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.523818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.523896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.523974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.524052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.524129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.524209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.524289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.524367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.524447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.524530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.524619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.524715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.524800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.524881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.524961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.525039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.525117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.525193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.525270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.525346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.525426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.525503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.525583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.525670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.525746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.525825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.525906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.525982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.526059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.526136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.526212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.526290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.526368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.526445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.526521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.526599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.526688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.526766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.526842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.526917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.526993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.527070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.527145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.527220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.527299] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.527423] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.527427] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38150, lrc_seqno=38150, guc_id=0, flags=0x73 in no process [-1]
<7> [245.527429] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.527497] ------------[ cut here ]------------
<4> [245.527498] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.527499] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:8/983
<4> [245.527589] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.527660]
<4> [245.527662] CPU: 4 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.527665] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.527666] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.527668] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.527673] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.527749] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.527750] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.527753] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.527754] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.527755] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.527757] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.527758] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.527759] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [245.527761] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.527762] CR2: 00007fe398459000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.527763] PKRU: 55555554
<4> [245.527765] Call Trace:
<4> [245.527766] <TASK>
<4> [245.527770] ? lock_acquire+0x60/0x300
<4> [245.527775] ? lock_release+0xd0/0x2b0
<4> [245.527779] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.527785] process_one_work+0x239/0x740
<4> [245.527792] worker_thread+0x200/0x3f0
<4> [245.527795] ? __pfx_worker_thread+0x10/0x10
<4> [245.527798] kthread+0x10d/0x150
<4> [245.527800] ? __pfx_kthread+0x10/0x10
<4> [245.527803] ret_from_fork+0x3bd/0x470
<4> [245.527805] ? __pfx_kthread+0x10/0x10
<4> [245.527808] ret_from_fork_asm+0x1a/0x30
<4> [245.527817] </TASK>
<4> [245.527818] irq event stamp: 1362673
<4> [245.527819] hardirqs last enabled at (1362679): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.527821] hardirqs last disabled at (1362684): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.527823] softirqs last enabled at (1361768): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.527826] softirqs last disabled at (1361743): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.527828] ---[ end trace 0000000000000000 ]---
<6> [245.528685] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.528800] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.528809] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.528841] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.528950] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.529062] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.529278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.529378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.529477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.529573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.529681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.529772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.529861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.529950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.530033] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.530133] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.530228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.531264] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.541641] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.541899] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.542883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.542965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.543045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.543124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.543202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.543280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.543357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.543434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.543510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.543587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.543674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.543750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.543826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.543901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.543977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.544051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.544126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.544201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.544277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.544363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.544446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.544526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.544603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.544690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.544771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.544848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.544923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.545000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.545076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.545154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.545232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.545310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.545388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.545464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.545547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.545630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.545727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.545806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.545885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.545960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.546036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.546113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.546188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.546265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.546341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.546418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.546493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.546568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.546647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.546724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.546804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.546880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.546956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.547030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.547107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.547183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.547259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.547334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.547407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.547485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.547561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.547642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.547718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.547793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.547870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.547947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.548021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.548097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.548175] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.548297] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.548302] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38150, lrc_seqno=38150, guc_id=0, flags=0x73 in no process [-1]
<7> [245.548304] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.548374] ------------[ cut here ]------------
<4> [245.548376] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.548377] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:8/983
<4> [245.548455] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.548524]
<4> [245.548526] CPU: 6 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.548529] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.548530] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.548532] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.548538] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.548613] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.548615] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.548617] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.548618] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.548620] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.548621] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.548622] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.548623] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [245.548625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.548626] CR2: 0000718d911e3000 CR3: 0000000136bff000 CR4: 0000000000f52ef0
<4> [245.548628] PKRU: 55555554
<4> [245.548629] Call Trace:
<4> [245.548630] <TASK>
<4> [245.548638] ? lock_acquire+0x60/0x300
<4> [245.548644] ? lock_release+0xd0/0x2b0
<4> [245.548649] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.548655] process_one_work+0x239/0x740
<4> [245.548662] worker_thread+0x200/0x3f0
<4> [245.548666] ? __pfx_worker_thread+0x10/0x10
<4> [245.548668] kthread+0x10d/0x150
<4> [245.548670] ? __pfx_kthread+0x10/0x10
<4> [245.548673] ret_from_fork+0x3bd/0x470
<4> [245.548676] ? __pfx_kthread+0x10/0x10
<4> [245.548678] ret_from_fork_asm+0x1a/0x30
<4> [245.548686] </TASK>
<4> [245.548687] irq event stamp: 1365833
<4> [245.548688] hardirqs last enabled at (1365839): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.548691] hardirqs last disabled at (1365844): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.548693] softirqs last enabled at (1364846): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.548696] softirqs last disabled at (1364839): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.548698] ---[ end trace 0000000000000000 ]---
<7> [245.550012] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.550176] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.551823] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [245.551931] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [245.552021] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.552103] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552182] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552259] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552336] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.552415] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552491] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552565] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.552660] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [245.552778] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [245.552891] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [245.553001] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [245.564293] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [245.648235] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.648834] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.649287] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.649754] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.650165] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.650562] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.651016] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.651407] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.651836] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.652227] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.652686] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.652847] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [245.652948] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [245.653044] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [245.698137] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38151, lrc_seqno=38151, guc_id=0, flags=0x73 in no process [-1]
<7> [245.698164] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.698590] ------------[ cut here ]------------
<4> [245.698596] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.698604] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:8/983
<4> [245.699153] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.699493]
<4> [245.699506] CPU: 2 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.699524] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.699531] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.699540] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.699573] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.700073] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.700083] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.700096] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.700105] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.700113] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.700120] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.700128] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.700136] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [245.700144] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.700153] CR2: 00007fe39845a000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.700162] PKRU: 55555554
<4> [245.700169] Call Trace:
<4> [245.700176] <TASK>
<4> [245.700198] ? lock_acquire+0x60/0x300
<4> [245.700227] ? lock_release+0xd0/0x2b0
<4> [245.700253] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.700284] process_one_work+0x239/0x740
<4> [245.700320] worker_thread+0x200/0x3f0
<4> [245.700336] ? __pfx_worker_thread+0x10/0x10
<4> [245.700351] kthread+0x10d/0x150
<4> [245.700365] ? __pfx_kthread+0x10/0x10
<4> [245.700380] ret_from_fork+0x3bd/0x470
<4> [245.700394] ? __pfx_kthread+0x10/0x10
<4> [245.700409] ret_from_fork_asm+0x1a/0x30
<4> [245.700449] </TASK>
<4> [245.700457] irq event stamp: 1366721
<4> [245.700464] hardirqs last enabled at (1366727): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.700476] hardirqs last disabled at (1366732): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.700486] softirqs last enabled at (1365976): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.700504] softirqs last disabled at (1365969): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.700519] ---[ end trace 0000000000000000 ]---
<6> [245.700529] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.701003] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.701067] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.701246] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.701873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.702391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.702956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.703239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.703328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.703415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.703502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.703590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.703683] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.703799] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.703891] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.704977] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.716012] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.716367] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.717599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.717786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.717939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.718090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.718244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.718396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.718548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.718712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.718867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.719024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.719185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.719341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.719497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.719659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.719812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.719959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.720098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.720238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.720376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.720539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.720708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.720859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.721002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.721143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.721278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.721406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.721534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.721670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.721801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.721927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.722054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.722172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.722290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.722403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.722522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.722643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.722763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.722873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.722983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.723086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.723190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.723295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.723394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.723495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.723596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.723705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.723800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.723894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.723986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.724077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.724170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.724256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.724344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.724430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.724513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.724600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.724694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.724779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.724864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.724949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.725034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.725115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.725199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.725278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.725360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.725441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.725521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.725600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.725692] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.725821] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.725825] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38151, lrc_seqno=38151, guc_id=0, flags=0x73 in no process [-1]
<7> [245.725828] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.725900] ------------[ cut here ]------------
<4> [245.725901] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.725902] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#2: kworker/u64:8/983
<4> [245.725986] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.726051]
<4> [245.726053] CPU: 2 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.726056] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.726057] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.726058] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.726064] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.726145] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.726147] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.726149] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.726150] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.726151] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.726153] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.726154] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.726155] FS: 0000000000000000(0000) GS:ffff8888dad83000(0000) knlGS:0000000000000000
<4> [245.726157] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.726158] CR2: 00007fe39845a000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.726160] PKRU: 55555554
<4> [245.726161] Call Trace:
<4> [245.726162] <TASK>
<4> [245.726166] ? lock_acquire+0x60/0x300
<4> [245.726171] ? lock_release+0xd0/0x2b0
<4> [245.726176] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.726182] process_one_work+0x239/0x740
<4> [245.726188] worker_thread+0x200/0x3f0
<4> [245.726191] ? __pfx_worker_thread+0x10/0x10
<4> [245.726194] kthread+0x10d/0x150
<4> [245.726196] ? __pfx_kthread+0x10/0x10
<4> [245.726199] ret_from_fork+0x3bd/0x470
<4> [245.726201] ? __pfx_kthread+0x10/0x10
<4> [245.726204] ret_from_fork_asm+0x1a/0x30
<4> [245.726212] </TASK>
<4> [245.726213] irq event stamp: 1369915
<4> [245.726214] hardirqs last enabled at (1369921): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.726216] hardirqs last disabled at (1369926): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.726218] softirqs last enabled at (1368986): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.726221] softirqs last disabled at (1368979): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.726223] ---[ end trace 0000000000000000 ]---
<6> [245.726225] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.726305] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.726311] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.727650] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.727765] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.727880] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.728097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.728193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.728290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.728382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.728473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.728561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.728663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.728753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.728835] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.728935] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.729026] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.730063] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.740794] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.741051] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.742115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.742201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.742282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.742362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.742439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.742517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.742593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.742680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.742758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.742833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.742908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.742982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.743056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.743131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.743206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.743280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.743356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.743429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.743505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.743590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.743683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.743769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.743849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.743929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.744010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.744087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.744164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.744241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.744318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.744398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.744482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.744565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.744651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.744733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.744818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.744903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.744987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.745068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.745148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.745225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.745304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.745381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.745456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.745534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.745613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.745701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.745779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.745857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.745932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.746009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.746088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.746164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.746240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.746315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.746391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.746467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.746542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.746616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.746701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.746780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.746856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.746933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.747010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.747090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.747170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.747249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.747326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.747402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.747481] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.747608] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.747613] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38151, lrc_seqno=38151, guc_id=0, flags=0x73 in no process [-1]
<7> [245.747615] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.747698] ------------[ cut here ]------------
<4> [245.747699] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.747701] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:8/983
<4> [245.747779] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.747844]
<4> [245.747847] CPU: 3 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.747850] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.747851] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.747852] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.747858] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.747934] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.747936] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.747938] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.747939] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.747940] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.747942] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.747943] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.747944] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [245.747946] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.747947] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.747949] PKRU: 55555554
<4> [245.747950] Call Trace:
<4> [245.747951] <TASK>
<4> [245.747955] ? lock_acquire+0x60/0x300
<4> [245.747960] ? lock_release+0xd0/0x2b0
<4> [245.747965] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.747971] process_one_work+0x239/0x740
<4> [245.747977] worker_thread+0x200/0x3f0
<4> [245.747981] ? __pfx_worker_thread+0x10/0x10
<4> [245.747983] kthread+0x10d/0x150
<4> [245.747985] ? __pfx_kthread+0x10/0x10
<4> [245.747988] ret_from_fork+0x3bd/0x470
<4> [245.747990] ? __pfx_kthread+0x10/0x10
<4> [245.747993] ret_from_fork_asm+0x1a/0x30
<4> [245.748001] </TASK>
<4> [245.748002] irq event stamp: 1373079
<4> [245.748003] hardirqs last enabled at (1373085): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.748005] hardirqs last disabled at (1373090): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.748007] softirqs last enabled at (1372076): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.748010] softirqs last disabled at (1372071): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.748012] ---[ end trace 0000000000000000 ]---
<7> [245.749318] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.749432] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [245.755142] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38159, lrc_seqno=38159, guc_id=0, flags=0x73 in no process [-1]
<7> [245.755145] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.755215] ------------[ cut here ]------------
<4> [245.755217] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.755218] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:8/983
<4> [245.755299] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.755363]
<4> [245.755365] CPU: 3 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.755367] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.755368] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.755370] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.755375] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.755452] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.755454] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.755456] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.755458] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.755459] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.755460] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.755461] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.755463] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [245.755464] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.755465] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.755467] PKRU: 55555554
<4> [245.755468] Call Trace:
<4> [245.755469] <TASK>
<4> [245.755473] ? lock_acquire+0x60/0x300
<4> [245.755478] ? lock_release+0xd0/0x2b0
<4> [245.755482] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.755488] process_one_work+0x239/0x740
<4> [245.755494] worker_thread+0x200/0x3f0
<4> [245.755498] ? __pfx_worker_thread+0x10/0x10
<4> [245.755500] kthread+0x10d/0x150
<4> [245.755502] ? __pfx_kthread+0x10/0x10
<4> [245.755505] ret_from_fork+0x3bd/0x470
<4> [245.755508] ? __pfx_kthread+0x10/0x10
<4> [245.755510] ret_from_fork_asm+0x1a/0x30
<4> [245.755518] </TASK>
<4> [245.755519] irq event stamp: 1374089
<4> [245.755520] hardirqs last enabled at (1374095): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.755522] hardirqs last disabled at (1374100): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.755524] softirqs last enabled at (1372076): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.755527] softirqs last disabled at (1372071): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.755529] ---[ end trace 0000000000000000 ]---
<6> [245.755531] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.755607] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.755614] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.755801] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.756011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.756107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.756203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.756294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.756384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.756472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.756561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.756655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.756740] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.756841] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.756935] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.757968] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.768976] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.769234] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.770231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.770317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.770397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.770477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.770555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.770638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.770716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.770792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.770868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.770945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.771021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.771098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.771174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.771250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.771325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.771401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.771473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.771546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.771620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.771716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.771801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.771883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.771962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.772042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.772121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.772197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.772273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.772352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.772428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.772506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.772590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.772677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.772758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.772837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.772922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.773007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.773091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.773174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.773257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.773338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.773417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.773492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.773569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.773651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.773731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.773808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.773886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.773963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.774041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.774119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.774198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.774275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.774352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.774431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.774506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.774581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.774662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.774737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.774812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.774890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.774965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.775042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.775117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.775191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.775268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.775343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.775416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.775490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.775567] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.775692] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.775696] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38159, lrc_seqno=38159, guc_id=0, flags=0x73 in no process [-1]
<7> [245.775698] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.775765] ------------[ cut here ]------------
<4> [245.775766] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.775767] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:8/983
<4> [245.775845] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.775908]
<4> [245.775910] CPU: 3 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.775913] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.775914] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.775915] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.775921] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.775997] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.775998] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.776000] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.776002] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.776003] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.776004] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.776006] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.776007] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [245.776008] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.776010] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.776011] PKRU: 55555554
<4> [245.776012] Call Trace:
<4> [245.776013] <TASK>
<4> [245.776017] ? lock_acquire+0x60/0x300
<4> [245.776022] ? lock_release+0xd0/0x2b0
<4> [245.776027] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.776033] process_one_work+0x239/0x740
<4> [245.776039] worker_thread+0x200/0x3f0
<4> [245.776042] ? __pfx_worker_thread+0x10/0x10
<4> [245.776045] kthread+0x10d/0x150
<4> [245.776047] ? __pfx_kthread+0x10/0x10
<4> [245.776050] ret_from_fork+0x3bd/0x470
<4> [245.776053] ? __pfx_kthread+0x10/0x10
<4> [245.776055] ret_from_fork_asm+0x1a/0x30
<4> [245.776063] </TASK>
<4> [245.776064] irq event stamp: 1377267
<4> [245.776065] hardirqs last enabled at (1377273): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.776067] hardirqs last disabled at (1377278): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.776069] softirqs last enabled at (1376424): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.776072] softirqs last disabled at (1376413): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.776074] ---[ end trace 0000000000000000 ]---
<6> [245.776076] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.776150] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.776156] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.777450] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.777564] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.777680] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.777891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.777982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.778077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.778166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.778252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.778338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.778427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.778514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.778595] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.778704] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.778798] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.779830] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.789641] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.789896] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.790903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.790988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.791069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.791149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.791227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.791305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.791382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.791458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.791535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.791610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.791695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.791771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.791846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.791922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.791997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.792070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.792145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.792220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.792294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.792377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.792460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.792542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.792624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.792721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.792802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.792881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.792960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.793037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.793118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.793199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.793280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.793359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.793438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.793516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.793600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.793693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.793778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.793859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.793938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.794015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.794100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.794180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.794257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.794335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.794413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.794491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.794568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.794648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.794724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.794801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.794880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.794957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.795039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.795120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.795197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.795275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.795350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.795426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.795500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.795579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.795662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.795741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.795817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.795892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.795970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.796047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.796122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.796198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.796275] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.796396] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.796400] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38159, lrc_seqno=38159, guc_id=0, flags=0x73 in no process [-1]
<7> [245.796402] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.796470] ------------[ cut here ]------------
<4> [245.796471] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.796472] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#3: kworker/u64:8/983
<4> [245.796560] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.796623]
<4> [245.796625] CPU: 3 UID: 0 PID: 983 Comm: kworker/u64:8 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.796628] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.796629] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.796630] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.796642] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.796720] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.796722] RSP: 0018:ffffc90003173ca0 EFLAGS: 00010246
<4> [245.796724] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.796725] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.796726] RBP: ffffc90003173db0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.796728] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.796729] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.796730] FS: 0000000000000000(0000) GS:ffff8888dae03000(0000) knlGS:0000000000000000
<4> [245.796732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.796733] CR2: 0000718d911e3000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.796735] PKRU: 55555554
<4> [245.796736] Call Trace:
<4> [245.796737] <TASK>
<4> [245.796741] ? lock_acquire+0x60/0x300
<4> [245.796745] ? lock_release+0xd0/0x2b0
<4> [245.796750] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.796756] process_one_work+0x239/0x740
<4> [245.796763] worker_thread+0x200/0x3f0
<4> [245.796766] ? __pfx_worker_thread+0x10/0x10
<4> [245.796768] kthread+0x10d/0x150
<4> [245.796770] ? __pfx_kthread+0x10/0x10
<4> [245.796773] ret_from_fork+0x3bd/0x470
<4> [245.796776] ? __pfx_kthread+0x10/0x10
<4> [245.796778] ret_from_fork_asm+0x1a/0x30
<4> [245.796785] </TASK>
<4> [245.796787] irq event stamp: 1380473
<4> [245.796788] hardirqs last enabled at (1380479): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.796790] hardirqs last disabled at (1380484): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.796792] softirqs last enabled at (1380300): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.796794] softirqs last disabled at (1380279): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.796797] ---[ end trace 0000000000000000 ]---
<7> [245.797813] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.797930] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.799421] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1515) -> ( 0 - 1989), size 1515 -> 1989
<7> [245.799525] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb (1515 - 1989) -> ( 0 - 0), size 474 -> 0
<7> [245.799612] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.799721] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.799806] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.799885] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.799961] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm -> wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm
<7> [245.800045] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.800120] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.800190] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46 -> 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0
<7> [245.800288] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 594000 num active planes 1
<7> [245.800408] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 624 qgv_peak_bw: 48000
<7> [245.800514] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 624 qgv_peak_bw: 48000
<7> [245.800623] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 624
<7> [245.814285] xe 0000:03:00.0: [drm:i915_fifo_underrun_reset_write [xe]] Re-arming FIFO underruns on pipe C
<7> [245.898304] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.898924] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.899374] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.899857] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.900465] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.901245] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.901848] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.902308] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.902758] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.903173] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.903672] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.904618] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:153:plane 1C] ddb ( 0 - 1989) -> ( 0 - 1515), size 1989 -> 1515
<7> [245.904930] xe 0000:03:00.0: [drm:skl_print_plane_ddb_changes [xe]] [PLANE:183:plane 2C] ddb ( 0 - 0) -> (1515 - 1989), size 0 -> 474
<7> [245.905006] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.905078] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 2, 3, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.905148] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 28, 82, 82, 154, 154, 154, 154, 28, 136, 150
<7> [245.905218] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:183:plane 2C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 32, 92, 92, 171, 171, 0, 0, 29, 151, 151
<7> [245.905288] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] level wm0, wm1, wm2, wm3, wm4, wm5, wm6, wm7, twm, swm, stwm -> *wm0,*wm1,*wm2,*wm3,*wm4,*wm5, wm6, wm7,*twm,*swm,*stwm
<7> [245.905355] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] lines 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 7, 7, 9, 9, 17, 17, 17, 17, 0, 15, 0
<7> [245.905425] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] blocks 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 14, 14, 19, 19, 35, 35, 35, 35, 28, 31, 45
<7> [245.905493] xe 0000:03:00.0: [drm:skl_print_plane_wm_changes [xe]] [PLANE:263:cursor C] min_ddb 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0 -> 17, 17, 22, 22, 40, 40, 0, 0, 29, 36, 46
<7> [245.905566] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] [CRTC:269:pipe C] data rate 1188000 num active planes 2
<7> [245.905681] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 0: max bw 33600 required 1248 qgv_peak_bw: 48000
<7> [245.905793] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] QGV point 1: max bw 53000 required 1248 qgv_peak_bw: 48000
<7> [245.905891] xe 0000:03:00.0: [drm:intel_bw_atomic_check [xe]] Matching peaks QGV bw: 48000 for required data rate: 1248
<5> [245.948444] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38160, lrc_seqno=38160, guc_id=0, flags=0x73 in no process [-1]
<7> [245.948470] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.949007] ------------[ cut here ]------------
<4> [245.949015] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.949023] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#4: kworker/u64:2/121
<4> [245.949501] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.949917]
<4> [245.949937] CPU: 4 UID: 0 PID: 121 Comm: kworker/u64:2 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.949960] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.949972] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.949984] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.950033] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.950578] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.950588] RSP: 0018:ffffc9000056fca0 EFLAGS: 00010246
<4> [245.950601] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.950609] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.950615] RBP: ffffc9000056fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.950622] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.950628] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.950651] FS: 0000000000000000(0000) GS:ffff8888dae83000(0000) knlGS:0000000000000000
<4> [245.950660] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.950667] CR2: 00007fe39845b000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.950675] PKRU: 55555554
<4> [245.950681] Call Trace:
<4> [245.950687] <TASK>
<4> [245.950709] ? lock_acquire+0x60/0x300
<4> [245.950737] ? lock_release+0xd0/0x2b0
<4> [245.950763] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.950795] process_one_work+0x239/0x740
<4> [245.950832] worker_thread+0x200/0x3f0
<4> [245.950848] ? __pfx_worker_thread+0x10/0x10
<4> [245.950862] kthread+0x10d/0x150
<4> [245.950873] ? __pfx_kthread+0x10/0x10
<4> [245.950888] ret_from_fork+0x3bd/0x470
<4> [245.950899] ? __pfx_kthread+0x10/0x10
<4> [245.950913] ret_from_fork_asm+0x1a/0x30
<4> [245.950952] </TASK>
<4> [245.950958] irq event stamp: 703261
<4> [245.950965] hardirqs last enabled at (703267): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.950978] hardirqs last disabled at (703272): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.950987] softirqs last enabled at (702494): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.951002] softirqs last disabled at (702487): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.951015] ---[ end trace 0000000000000000 ]---
<6> [245.951026] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.951435] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.951467] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.951744] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.952435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.953063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.953593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.954142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.954611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.955122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.955580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.956103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.956558] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.957195] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.957743] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.959534] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.970776] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.971092] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.972134] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.972247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.972350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.972449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.972544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.972646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.972742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.972837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.972937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.973036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.973131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.973227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.973321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.973414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.973505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.973596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.973701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.973792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.973875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.973974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.974068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.974157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.974245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.974333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.974421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.974508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.974593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.974685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.974768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.974852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.974939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.975023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.975107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.975191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.975281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.975369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.975457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.975543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.975628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.975722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.975806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.975888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.975970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.976052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.976136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.976219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.976302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.976386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.976468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.976550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
<7> [245.976640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [245.976722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [245.976805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a098] = 0xffffffff
<7> [245.976887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [245.976968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [245.977050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a29c] = 0x00080008
<7> [245.977130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [245.977210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [245.977291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [245.977374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [245.977455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [245.977536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c098] = 0xffffffff
<7> [245.977618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [245.977711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [245.977797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c29c] = 0x00080008
<7> [245.977878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [245.977960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [245.978039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [245.978121] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [245.978248] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [245.978253] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=38160, lrc_seqno=38160, guc_id=0, flags=0x73 in no process [-1]
<7> [245.978256] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [245.978328] ------------[ cut here ]------------
<4> [245.978330] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [245.978331] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1618 at guc_exec_queue_timedout_job+0x141a/0x2400 [xe], CPU#6: kworker/u64:4/203
<4> [245.978416] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy gpu_sched drm_ttm_helper ttm drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel snd_hda_intel cmdlinepart snd_intel_dspcfg kvm hid_generic asus_nb_wmi snd_hda_codec spi_nor asus_wmi snd_hda_core sparse_keymap mtd wmi_bmof irqbypass snd_hwdep aesni_intel gf128mul r8169 snd_pcm usbhid video rapl mei_pxp mei_hdcp binfmt_misc intel_cstate hid i2c_i801 snd_timer realtek phy_package i2c_mux spi_intel_pci snd i2c_smbus spi_intel soundcore nls_iso8859_1 intel_pmc_core pmt_telemetry pmt_discovery pmt_class idma64 intel_pmc_ssram_telemetry acpi_tad pinctrl_alderlake acpi_pad mei_me mei wmi intel_vsec dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [245.978482]
<4> [245.978485] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.1.0-rc3-lgci-xe-xe-5043-0ce1c813197dfbe15-debug+ #1 PREEMPT(lazy)
<4> [245.978488] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4> [245.978489] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 0812 02/24/2023
<4> [245.978490] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [245.978496] RIP: 0010:guc_exec_queue_timedout_job+0x1423/0x2400 [xe]
<4> [245.978584] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 20 d2 3c e1 48 89 c6 48 8d 3d b6 8d d9 ff 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 d0 ee ff ff 8b 70 08 49
<4> [245.978586] RSP: 0018:ffffc9000162fca0 EFLAGS: 00010246
<4> [245.978588] RAX: ffffffffa12bbe79 RBX: 0000000000000000 RCX: 0000000000000000
<4> [245.978590] RDX: ffff888103f8e590 RSI: ffffffffa12bbe79 RDI: ffffffffa0c03f80
<4> [245.978591] RBP: ffffc9000162fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [245.978592] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [245.978593] R13: ffff888103f8e590 R14: ffff888123eeb818 R15: 00000000ffffffc2
<4> [245.978595] FS: 0000000000000000(0000) GS:ffff8888daf83000(0000) knlGS:0000000000000000
<4> [245.978596] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [245.978598] CR2: 00007fe39845c000 CR3: 000000000344a000 CR4: 0000000000f52ef0
<4> [245.978599] PKRU: 55555554
<4> [245.978600] Call Trace:
<4> [245.978601] <TASK>
<4> [245.978606] ? lock_acquire+0x60/0x300
<4> [245.978611] ? lock_release+0xd0/0x2b0
<4> [245.978616] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [245.978622] process_one_work+0x239/0x740
<4> [245.978629] worker_thread+0x200/0x3f0
<4> [245.978635] ? __pfx_worker_thread+0x10/0x10
<4> [245.978638] kthread+0x10d/0x150
<4> [245.978641] ? __pfx_kthread+0x10/0x10
<4> [245.978644] ret_from_fork+0x3bd/0x470
<4> [245.978646] ? __pfx_kthread+0x10/0x10
<4> [245.978649] ret_from_fork_asm+0x1a/0x30
<4> [245.978657] </TASK>
<4> [245.978658] irq event stamp: 227763
<4> [245.978659] hardirqs last enabled at (227769): [<ffffffff814ab359>] __up_console_sem+0x79/0xa0
<4> [245.978661] hardirqs last disabled at (227774): [<ffffffff814ab33e>] __up_console_sem+0x5e/0xa0
<4> [245.978663] softirqs last enabled at (226878): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.978666] softirqs last disabled at (226867): [<ffffffff813d230b>] __irq_exit_rcu+0xdb/0x1c0
<4> [245.978669] ---[ end trace 0000000000000000 ]---
<6> [245.979619] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [245.979822] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [245.979834] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [245.979913] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [245.980039] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [245.980159] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [245.980393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [245.980501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [245.980606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [245.980714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [245.980809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [245.980901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [245.980993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [245.981089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [245.981182] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [245.981289] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [245.981390] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [245.982429] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [245.992641] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [245.992907] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [245.993903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [245.993996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [245.994083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [245.994171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [245.994257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [245.994340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [245.994422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [245.994504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [245.994586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [245.994676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [245.994758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [245.994839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [245.994920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [245.995001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [245.995080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [245.995162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [245.995243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [245.995324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [245.995403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [245.995493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [245.995580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2098] = 0xffffffff
<7> [245.995678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [245.995763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [245.995849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x229c] = 0x00080008
<7> [245.995933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [245.996016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [245.996098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [245.996181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [245.996262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [245.996347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [245.996435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [245.996522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [245.996609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [245.996703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [245.996793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [245.996882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [245.996971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [245.997056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00030003
<7> [245.997141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [245.997223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [245.997309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22098] = 0xffffffff
<7> [245.997395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [245.997475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [245.997556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2229c] = 0x00080008
<7> [245.997643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [245.997730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [245.997812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee098] = 0xffffffff
<7> [245.997902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [245.997982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [245.998065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee29c] = 0x00080008
|