Result: 416 Warning(s)
i915_display_info21 igt_runner21 results21.json results21-xe-load.json guc_logs21.tar i915_display_info_post_exec21 boot21 dmesg21
| Detail | Value |
|---|---|
| Duration | unknown |
| Hostname |
shard-bmg-7 |
| Igt-Version |
IGT-Version: 2.4-g20d275e9b (x86_64) (Linux: 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ x86_64) |
| Out |
Using IGT_SRANDOM=1776327525 for randomisation Opened device: /dev/dri/card0 Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298: Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10) Last errno: 2, No such file or directory Starting subtest: pt-caching-update-pat-and-pte va_bits: 48, num_objs: 16384, spread over pt levels: [0]: 16384 [1]: 512 [2]: 16 Subtest pt-caching-update-pat-and-pte: SUCCESS (18.504s) runner: This test was killed due to a kernel taint (0x244). This test caused an abort condition: Kernel badly tainted (0x244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened. |
| Err |
Starting subtest: pt-caching-update-pat-and-pte Subtest pt-caching-update-pat-and-pte: SUCCESS (18.504s) |
| Dmesg |
<6> [174.981496] Console: switching to colour dummy device 80x25
<6> [174.981803] [IGT] xe_pat: executing
<6> [175.011720] [IGT] xe_pat: starting subtest pt-caching-update-pat-and-pte
<7> [176.474054] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling AUX_TC2
<7> [176.577939] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling AUX_TC2
<6> [176.853001] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x000000000060b000
FaultType: 0
AccessType: 1
FaultLevel: 0
EngineClass: 3 bcs
EngineInstance: 8
<6> [176.853014] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<6> [176.853064] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<7> [176.853831] xe 0000:03:00.0: [drm:xe_hw_engine_snapshot_capture [xe]] Tile0: GT0: Proceeding with manual engine snapshot
<5> [176.854127] xe 0000:03:00.0: [drm] Tile0: GT0: log buffer overflow
<7> [177.375249] xe 0000:03:00.0: [drm:xe_guc_capture_process [xe]] Tile0: GT0: Capture extraction failed:-5
<6> [177.375473] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [177.375606] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53474, lrc_seqno=53474, guc_id=0, flags=0x73 in no process [-1]
<6> [177.554318] xe 0000:03:00.0: [drm] Xe device coredump has been created
<6> [177.554337] xe 0000:03:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data
<4> [177.554340] ------------[ cut here ]------------
<4> [177.554342] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.554343] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:45/3063
<4> [177.554447] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.554559] efi_pstore nfnetlink autofs4
<4> [177.554563] CPU: 4 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.554567] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER
<4> [177.554568] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.554570] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.554576] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.554645] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.554646] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.554649] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.554650] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.554651] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.554652] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.554653] R13: ffff8881048d8c90 R14: ffff888111038000 R15: 00000000fffffffb
<4> [177.554655] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [177.554656] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.554657] CR2: 00007a396ca9c6f4 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [177.554659] PKRU: 55555554
<4> [177.554660] Call Trace:
<4> [177.554661] <TASK>
<4> [177.554665] ? lock_sync+0x100/0x100
<4> [177.554670] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [177.554675] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.554680] process_one_work+0x239/0x760
<4> [177.554687] worker_thread+0x200/0x3f0
<4> [177.554689] ? __pfx_worker_thread+0x10/0x10
<4> [177.554692] kthread+0x10d/0x150
<4> [177.554695] ? __pfx_kthread+0x10/0x10
<4> [177.554698] ret_from_fork+0x3d4/0x480
<4> [177.554701] ? __pfx_kthread+0x10/0x10
<4> [177.554704] ret_from_fork_asm+0x1a/0x30
<4> [177.554711] </TASK>
<4> [177.554712] irq event stamp: 469259
<4> [177.554713] hardirqs last enabled at (469265): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.554716] hardirqs last disabled at (469270): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.554718] softirqs last enabled at (468406): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.554721] softirqs last disabled at (468383): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.554723] ---[ end trace 0000000000000000 ]---
<6> [177.554725] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.554794] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.555074] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.555535] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.556000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.556135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.556229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.556316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.556401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.556485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.556569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.556652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.556732] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.556828] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.556952] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.558051] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.567831] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.568165] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.570020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.570111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.570190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.570266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.570340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.570413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.570487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.570566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.570646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.570723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.570801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.570899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.570973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.571047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.571125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.571201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.571271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.571339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.571407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.571483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.571557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.571635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.571716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.571796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.571890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.571966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.572039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.572116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.572192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.572265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.572337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.572410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.572488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.572565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.572644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.572719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.572795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.572885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.572963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.573035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.573108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.573179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.573250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.573320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.573399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.573477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.573555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.573634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.573717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.573797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.573893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.573973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.574052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.574132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.574214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.574294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.574373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.574452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.574534] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.575097] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.575106] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53474, lrc_seqno=53474, guc_id=0, flags=0x73 in no process [-1]
<7> [177.575112] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.575207] ------------[ cut here ]------------
<4> [177.575209] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.575212] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.575323] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.575454] efi_pstore nfnetlink autofs4
<4> [177.575463] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.575468] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.575470] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.575473] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.575482] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.575569] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.575570] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.575573] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.575575] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.575576] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.575577] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.575579] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.575580] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.575582] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.575583] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.575585] PKRU: 55555554
<4> [177.575586] Call Trace:
<4> [177.575587] <TASK>
<4> [177.575591] ? lock_sync+0x100/0x100
<4> [177.575597] ? lock_release+0xd0/0x2b0
<4> [177.575603] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.575609] process_one_work+0x239/0x760
<4> [177.575616] worker_thread+0x200/0x3f0
<4> [177.575619] ? __pfx_worker_thread+0x10/0x10
<4> [177.575622] kthread+0x10d/0x150
<4> [177.575625] ? __pfx_kthread+0x10/0x10
<4> [177.575629] ret_from_fork+0x3d4/0x480
<4> [177.575631] ? __pfx_kthread+0x10/0x10
<4> [177.575635] ret_from_fork_asm+0x1a/0x30
<4> [177.575642] </TASK>
<4> [177.575644] irq event stamp: 472401
<4> [177.575645] hardirqs last enabled at (472407): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.575648] hardirqs last disabled at (472412): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.575650] softirqs last enabled at (471600): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.575653] softirqs last disabled at (471593): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.575655] ---[ end trace 0000000000000000 ]---
<6> [177.575657] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.575730] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.575737] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.577069] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.577210] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.577422] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.577633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.577727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.577836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.577929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.578009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.578089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.578168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.578252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.578331] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.578420] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.578530] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.579551] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.589816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.590065] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.591147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.591219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.591291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.591362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.591432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.591500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.591568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.591636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.591703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.591770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.591857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.591940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.592010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.592079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.592147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.592215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.592283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.592351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.592418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.592492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.592568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.592641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.592715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.592785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.592872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.592957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.593026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.593098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.593172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.593247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.593323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.593397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.593475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.593554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.593632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.593705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.593779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.593866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.593954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.594027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.594103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.594175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.594246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.594317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.594390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.594461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.594535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.594607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.594680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.594749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.594825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.594916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.594987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.595057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.595127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.595199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.595269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.595338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.595410] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.595514] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.595522] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53474, lrc_seqno=53474, guc_id=0, flags=0x73 in no process [-1]
<7> [177.595529] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.595735] ------------[ cut here ]------------
<4> [177.595738] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.595740] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#14: kworker/u64:39/3057
<4> [177.595872] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.595974] efi_pstore nfnetlink autofs4
<4> [177.595982] CPU: 14 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.595986] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.595988] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.595990] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.595999] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.596125] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.596127] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.596130] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.596133] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.596134] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.596136] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.596138] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.596140] FS: 0000000000000000(0000) GS:ffff8888db397000(0000) knlGS:0000000000000000
<4> [177.596143] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.596145] CR2: 000061025014c6b8 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [177.596147] PKRU: 55555554
<4> [177.596149] Call Trace:
<4> [177.596150] <TASK>
<4> [177.596157] ? lock_sync+0x100/0x100
<4> [177.596166] ? lock_release+0xd0/0x2b0
<4> [177.596175] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.596184] process_one_work+0x239/0x760
<4> [177.596196] worker_thread+0x200/0x3f0
<4> [177.596200] ? __pfx_worker_thread+0x10/0x10
<4> [177.596203] kthread+0x10d/0x150
<4> [177.596208] ? __pfx_kthread+0x10/0x10
<4> [177.596213] ret_from_fork+0x3d4/0x480
<4> [177.596218] ? __pfx_kthread+0x10/0x10
<4> [177.596223] ret_from_fork_asm+0x1a/0x30
<4> [177.596236] </TASK>
<4> [177.596238] irq event stamp: 97465
<4> [177.596240] hardirqs last enabled at (97471): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.596244] hardirqs last disabled at (97476): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.596248] softirqs last enabled at (96722): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.596252] softirqs last disabled at (96713): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.596255] ---[ end trace 0000000000000000 ]---
<7> [177.596526] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.596677] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [177.597515] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53529, lrc_seqno=53529, guc_id=0, flags=0x73 in no process [-1]
<7> [177.597520] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.597624] ------------[ cut here ]------------
<4> [177.597625] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.597627] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#14: kworker/u64:39/3057
<4> [177.597741] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.597838] efi_pstore nfnetlink autofs4
<4> [177.597844] CPU: 14 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.597848] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.597849] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.597851] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.597858] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.597972] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.597974] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.597977] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.597979] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.597981] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.597983] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.597984] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.597986] FS: 0000000000000000(0000) GS:ffff8888db397000(0000) knlGS:0000000000000000
<4> [177.597988] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.597990] CR2: 000061025014c6b8 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [177.597992] PKRU: 55555554
<4> [177.597994] Call Trace:
<4> [177.597996] <TASK>
<4> [177.598002] ? lock_sync+0x100/0x100
<4> [177.598009] ? lock_release+0xd0/0x2b0
<4> [177.598017] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.598026] process_one_work+0x239/0x760
<4> [177.598035] worker_thread+0x200/0x3f0
<4> [177.598039] ? __pfx_worker_thread+0x10/0x10
<4> [177.598043] kthread+0x10d/0x150
<4> [177.598047] ? __pfx_kthread+0x10/0x10
<4> [177.598052] ret_from_fork+0x3d4/0x480
<4> [177.598054] ? __pfx_kthread+0x10/0x10
<4> [177.598059] ret_from_fork_asm+0x1a/0x30
<4> [177.598071] </TASK>
<4> [177.598073] irq event stamp: 99847
<4> [177.598074] hardirqs last enabled at (99853): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.598078] hardirqs last disabled at (99858): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.598081] softirqs last enabled at (99484): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.598084] softirqs last disabled at (99477): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.598087] ---[ end trace 0000000000000000 ]---
<6> [177.598089] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.598202] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.598212] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.598272] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.598526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.598660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.598797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.598936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.599070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.599202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.599336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.599470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.599600] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.599742] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.599911] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.601245] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.611816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.612130] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.613288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.613410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.613529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.613650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.613770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.613899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.614020] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.614140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.614260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.614380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.614500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.614620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.614740] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.614866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.614987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.615107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.615227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.615347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.615468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.615592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.615718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.615846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.615973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.616100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.616223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.616347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.616471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.616598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.616726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.616856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.616984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.617110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.617242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.617374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.617505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.617634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.617762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.617892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.618018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.618142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.618270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.618396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.618521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.618645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.618774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.618916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.619022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.619114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.619205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.619291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.619378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.619468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.619554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.619637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.619720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.619810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.619893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.619974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.620061] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.620178] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.620182] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53529, lrc_seqno=53529, guc_id=0, flags=0x73 in no process [-1]
<7> [177.620185] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.620247] ------------[ cut here ]------------
<4> [177.620248] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.620250] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:39/3057
<4> [177.620326] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.620392] efi_pstore nfnetlink autofs4
<4> [177.620396] CPU: 2 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.620399] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.620400] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.620402] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.620407] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.620481] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.620483] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.620486] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.620487] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.620488] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.620490] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.620491] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.620492] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [177.620494] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.620495] CR2: 000077a5ba09f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.620497] PKRU: 55555554
<4> [177.620498] Call Trace:
<4> [177.620499] <TASK>
<4> [177.620503] ? lock_sync+0x100/0x100
<4> [177.620508] ? lock_release+0xd0/0x2b0
<4> [177.620514] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.620520] process_one_work+0x239/0x760
<4> [177.620526] worker_thread+0x200/0x3f0
<4> [177.620529] ? __pfx_worker_thread+0x10/0x10
<4> [177.620531] kthread+0x10d/0x150
<4> [177.620534] ? __pfx_kthread+0x10/0x10
<4> [177.620538] ret_from_fork+0x3d4/0x480
<4> [177.620540] ? __pfx_kthread+0x10/0x10
<4> [177.620543] ret_from_fork_asm+0x1a/0x30
<4> [177.620551] </TASK>
<4> [177.620552] irq event stamp: 102947
<4> [177.620553] hardirqs last enabled at (102953): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.620556] hardirqs last disabled at (102958): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.620558] softirqs last enabled at (102090): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.620561] softirqs last disabled at (102083): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.620563] ---[ end trace 0000000000000000 ]---
<6> [177.620565] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.620642] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.620648] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.621893] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.622006] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.622119] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.622336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.622436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.622534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.622628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.622720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.622815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.622910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.623004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.623092] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.623195] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.623318] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.624333] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.634809] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.635060] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.636077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.636157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.636235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.636313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.636391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.636467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.636544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.636619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.636694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.636769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.636867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.636961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.637039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.637115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.637191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.637266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.637340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.637414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.637489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.637571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.637652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.637731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.637818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.637911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.637990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.638071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.638150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.638231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.638312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.638390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.638469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.638548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.638636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.638724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.638816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.638922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.639004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.639082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.639160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.639236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.639318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.639400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.639480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.639558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.639641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.639718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.639796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.639889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.639985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.640060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.640136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.640214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.640292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.640368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.640443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.640522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.640597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.640673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.640752] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.640873] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.640880] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53529, lrc_seqno=53529, guc_id=0, flags=0x73 in no process [-1]
<7> [177.640884] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.640945] ------------[ cut here ]------------
<4> [177.640946] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.640947] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.641020] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.641085] efi_pstore nfnetlink autofs4
<4> [177.641090] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.641092] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.641094] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.641095] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.641100] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.641171] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.641172] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.641175] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.641176] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.641178] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.641179] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.641180] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.641181] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.641183] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.641184] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.641186] PKRU: 55555554
<4> [177.641187] Call Trace:
<4> [177.641188] <TASK>
<4> [177.641192] ? lock_sync+0x100/0x100
<4> [177.641197] ? lock_release+0xd0/0x2b0
<4> [177.641202] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.641208] process_one_work+0x239/0x760
<4> [177.641214] worker_thread+0x200/0x3f0
<4> [177.641217] ? __pfx_worker_thread+0x10/0x10
<4> [177.641219] kthread+0x10d/0x150
<4> [177.641222] ? __pfx_kthread+0x10/0x10
<4> [177.641226] ret_from_fork+0x3d4/0x480
<4> [177.641228] ? __pfx_kthread+0x10/0x10
<4> [177.641231] ret_from_fork_asm+0x1a/0x30
<4> [177.641238] </TASK>
<4> [177.641240] irq event stamp: 478099
<4> [177.641241] hardirqs last enabled at (478105): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.641244] hardirqs last disabled at (478110): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.641246] softirqs last enabled at (477264): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.641248] softirqs last disabled at (477257): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.641250] ---[ end trace 0000000000000000 ]---
<5> [177.642243] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53584, lrc_seqno=53584, guc_id=0, flags=0x73 in no process [-1]
<7> [177.642250] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.642377] ------------[ cut here ]------------
<4> [177.642379] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.642381] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.642477] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.642554] efi_pstore nfnetlink autofs4
<4> [177.642560] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.642564] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.642565] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.642567] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.642575] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.642656] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.642658] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.642661] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.642663] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.642664] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.642665] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.642666] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.642668] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.642669] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.642671] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.642673] PKRU: 55555554
<4> [177.642674] Call Trace:
<4> [177.642675] <TASK>
<4> [177.642681] ? lock_sync+0x100/0x100
<4> [177.642687] ? lock_release+0xd0/0x2b0
<4> [177.642694] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.642699] process_one_work+0x239/0x760
<4> [177.642706] worker_thread+0x200/0x3f0
<4> [177.642709] ? __pfx_worker_thread+0x10/0x10
<4> [177.642712] kthread+0x10d/0x150
<4> [177.642715] ? __pfx_kthread+0x10/0x10
<4> [177.642718] ret_from_fork+0x3d4/0x480
<4> [177.642721] ? __pfx_kthread+0x10/0x10
<4> [177.642726] ret_from_fork_asm+0x1a/0x30
<4> [177.642733] </TASK>
<4> [177.642735] irq event stamp: 2399597
<4> [177.642736] hardirqs last enabled at (2399603): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.642740] hardirqs last disabled at (2399608): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.642742] softirqs last enabled at (2398028): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.642745] softirqs last disabled at (2398021): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.642748] ---[ end trace 0000000000000000 ]---
<6> [177.642751] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.642858] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.643071] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.643172] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.643282] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.643585] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.643791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.643900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.644008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.644100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.644189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.644274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.644360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.644448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.644531] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.644625] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.644745] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.645761] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.655808] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.656058] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.657133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.657213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.657291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.657370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.657447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.657527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.657603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.657679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.657754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.657837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.657925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.658000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.658073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.658148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.658223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.658298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.658374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.658452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.658528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.658612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.658695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.658775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.658869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.658970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.659052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.659133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.659212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.659294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.659375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.659454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.659537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.659621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.659708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.659794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.659896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.659993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.660075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.660153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.660230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.660307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.660388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.660466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.660544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.660621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.660706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.660788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.660880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.660973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.661052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.661128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.661205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.661284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.661361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.661439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.661515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.661593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.661669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.661744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.661830] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.661963] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.661967] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53584, lrc_seqno=53584, guc_id=0, flags=0x73 in no process [-1]
<7> [177.661970] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.662028] ------------[ cut here ]------------
<4> [177.662029] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.662031] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.662104] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.662168] efi_pstore nfnetlink autofs4
<4> [177.662172] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.662175] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.662177] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.662178] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.662183] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.662253] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.662254] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.662256] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.662258] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.662259] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.662260] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.662262] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.662263] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.662265] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.662266] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.662268] PKRU: 55555554
<4> [177.662269] Call Trace:
<4> [177.662270] <TASK>
<4> [177.662274] ? lock_sync+0x100/0x100
<4> [177.662278] ? lock_release+0xd0/0x2b0
<4> [177.662284] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.662289] process_one_work+0x239/0x760
<4> [177.662295] worker_thread+0x200/0x3f0
<4> [177.662298] ? __pfx_worker_thread+0x10/0x10
<4> [177.662300] kthread+0x10d/0x150
<4> [177.662303] ? __pfx_kthread+0x10/0x10
<4> [177.662307] ret_from_fork+0x3d4/0x480
<4> [177.662309] ? __pfx_kthread+0x10/0x10
<4> [177.662312] ret_from_fork_asm+0x1a/0x30
<4> [177.662319] </TASK>
<4> [177.662321] irq event stamp: 483489
<4> [177.662322] hardirqs last enabled at (483495): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.662324] hardirqs last disabled at (483500): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.662327] softirqs last enabled at (482386): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.662329] softirqs last disabled at (482377): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.662331] ---[ end trace 0000000000000000 ]---
<6> [177.662333] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.662403] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.662409] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.663579] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.663685] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.663791] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.664018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.664114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.664213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.664310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.664402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.664493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.664584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.664675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.664762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.664871] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.664994] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.666006] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.676733] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.676992] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.678028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.678115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.678197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.678279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.678361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.678441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.678521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.678600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.678679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.678758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.678848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.678927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.679010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.679091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.679173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.679253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.679332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.679410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.679489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.679577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.679663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.679748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.679845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.679930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.680013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.680097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.680179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.680269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.680356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.680441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.680526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.680611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.680700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.680789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.680889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.680976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.681062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.681150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.681235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.681317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.681403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.681487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.681570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.681651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.681736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.681827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.681913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.681997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.682082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.682166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.682250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.682337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.682420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.682502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.682584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.682668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.682749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.682839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.682923] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.683038] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.683042] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53584, lrc_seqno=53584, guc_id=0, flags=0x73 in no process [-1]
<7> [177.683045] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.683107] ------------[ cut here ]------------
<4> [177.683108] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.683109] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:39/3057
<4> [177.683186] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.683251] efi_pstore nfnetlink autofs4
<4> [177.683256] CPU: 2 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.683258] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.683260] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.683261] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.683266] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.683341] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.683343] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.683346] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.683347] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.683348] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.683349] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.683351] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.683352] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000
<4> [177.683354] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.683355] CR2: 000077a5ba09f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.683357] PKRU: 55555554
<4> [177.683358] Call Trace:
<4> [177.683359] <TASK>
<4> [177.683363] ? lock_sync+0x100/0x100
<4> [177.683368] ? lock_release+0xd0/0x2b0
<4> [177.683373] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.683379] process_one_work+0x239/0x760
<4> [177.683385] worker_thread+0x200/0x3f0
<4> [177.683388] ? __pfx_worker_thread+0x10/0x10
<4> [177.683390] kthread+0x10d/0x150
<4> [177.683393] ? __pfx_kthread+0x10/0x10
<4> [177.683397] ret_from_fork+0x3d4/0x480
<4> [177.683399] ? __pfx_kthread+0x10/0x10
<4> [177.683402] ret_from_fork_asm+0x1a/0x30
<4> [177.683410] </TASK>
<4> [177.683411] irq event stamp: 106437
<4> [177.683412] hardirqs last enabled at (106443): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.683415] hardirqs last disabled at (106448): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.683417] softirqs last enabled at (105588): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.683419] softirqs last disabled at (105583): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.683422] ---[ end trace 0000000000000000 ]---
<5> [177.684839] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53639, lrc_seqno=53639, guc_id=0, flags=0x73 in no process [-1]
<7> [177.684849] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.684978] ------------[ cut here ]------------
<4> [177.684979] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.684981] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [177.685061] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid
<7> [177.685048] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [177.685120] kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4
<4> [177.685177] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.685181] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.685183] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.685185] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.685196] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<7> [177.685194] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [177.685311] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.685314] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.685317] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.685319] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.685321] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.685323] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.685324] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.685326] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [177.685329] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.685331] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [177.685333] PKRU: 55555554
<4> [177.685334] Call Trace:
<4> [177.685336] <TASK>
<4> [177.685343] ? lock_sync+0x100/0x100
<4> [177.685350] ? lock_release+0xd0/0x2b0
<4> [177.685356] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.685362] process_one_work+0x239/0x760
<4> [177.685368] worker_thread+0x200/0x3f0
<4> [177.685371] ? __pfx_worker_thread+0x10/0x10
<4> [177.685374] kthread+0x10d/0x150
<4> [177.685376] ? __pfx_kthread+0x10/0x10
<4> [177.685380] ret_from_fork+0x3d4/0x480
<4> [177.685382] ? __pfx_kthread+0x10/0x10
<4> [177.685385] ret_from_fork_asm+0x1a/0x30
<4> [177.685393] </TASK>
<4> [177.685394] irq event stamp: 109173
<4> [177.685395] hardirqs last enabled at (109179): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.685398] hardirqs last disabled at (109184): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.685400] softirqs last enabled at (105588): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.685402] softirqs last disabled at (105583): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.685404] ---[ end trace 0000000000000000 ]---
<6> [177.685406] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.685476] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.685484] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.685614] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.685826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.685916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.686010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.686097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.686182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.686266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.686352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.686439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.686526] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.686624] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.686742] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.687757] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.698491] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.698742] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.699769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.699860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.699938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.700017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.700095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.700172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.700248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.700327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.700406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.700484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.700560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.700636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.700711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.700785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.700874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.700948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.701022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.701096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.701171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.701252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.701338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.701421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.701505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.701584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.701663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.701744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.701835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.701917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.702003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.702085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.702166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.702246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.702331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.702417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.702500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.702583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.702667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.702749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.702837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.702915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.702996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.703076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.703154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.703231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.703313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.703391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.703476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.703555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.703634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.703711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.703787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.703880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.703958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.704036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.704117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.704198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.704275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.704353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.704433] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.704545] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.704549] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53639, lrc_seqno=53639, guc_id=0, flags=0x73 in no process [-1]
<7> [177.704552] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.704611] ------------[ cut here ]------------
<4> [177.704612] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.704613] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [177.704686] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.704750] efi_pstore nfnetlink autofs4
<4> [177.704754] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.704757] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.704758] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.704759] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.704765] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.704844] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.704846] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.704849] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.704850] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.704851] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.704853] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.704854] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.704856] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [177.704858] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.704859] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [177.704861] PKRU: 55555554
<4> [177.704862] Call Trace:
<4> [177.704863] <TASK>
<4> [177.704867] ? lock_sync+0x100/0x100
<4> [177.704871] ? lock_release+0xd0/0x2b0
<4> [177.704877] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.704882] process_one_work+0x239/0x760
<4> [177.704888] worker_thread+0x200/0x3f0
<4> [177.704891] ? __pfx_worker_thread+0x10/0x10
<4> [177.704893] kthread+0x10d/0x150
<4> [177.704896] ? __pfx_kthread+0x10/0x10
<4> [177.704900] ret_from_fork+0x3d4/0x480
<4> [177.704902] ? __pfx_kthread+0x10/0x10
<4> [177.704905] ret_from_fork_asm+0x1a/0x30
<4> [177.704913] </TASK>
<4> [177.704914] irq event stamp: 112273
<4> [177.704915] hardirqs last enabled at (112279): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.704918] hardirqs last disabled at (112284): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.704920] softirqs last enabled at (111108): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.704923] softirqs last disabled at (111103): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.704925] ---[ end trace 0000000000000000 ]---
<6> [177.704927] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.704998] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.705004] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.706244] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.706350] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.706457] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.706666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.706761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.706895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.706986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.707075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.707163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.707255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.707346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.707432] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.707529] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.707646] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.708660] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.719383] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.719635] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.720649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.720730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.720815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.720899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.720976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.721053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.721132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.721209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.721286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.721362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.721437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.721512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.721585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.721659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.721733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.721812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.721902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.721976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.722049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.722129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.722211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.722293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.722376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.722456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.722535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.722613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.722691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.722771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.722864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.722944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.723026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.723110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.723196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.723282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.723367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.723448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.723529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.723608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.723685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.723762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.723852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.723931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.724009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.724086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.724173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.724255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.724335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.724413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.724495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.724575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.724654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.724735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.724820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.724900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.724977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.725055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.725131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.725207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.725287] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.725398] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.725402] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53639, lrc_seqno=53639, guc_id=0, flags=0x73 in no process [-1]
<7> [177.725405] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.725464] ------------[ cut here ]------------
<4> [177.725465] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.725467] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.725539] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.725603] efi_pstore nfnetlink autofs4
<4> [177.725608] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.725611] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.725612] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.725614] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.725619] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.725689] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.725691] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.725693] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.725694] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.725696] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.725697] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.725698] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.725700] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.725701] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.725703] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.725704] PKRU: 55555554
<4> [177.725705] Call Trace:
<4> [177.725707] <TASK>
<4> [177.725711] ? lock_sync+0x100/0x100
<4> [177.725715] ? lock_release+0xd0/0x2b0
<4> [177.725721] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.725726] process_one_work+0x239/0x760
<4> [177.725733] worker_thread+0x200/0x3f0
<4> [177.725735] ? __pfx_worker_thread+0x10/0x10
<4> [177.725738] kthread+0x10d/0x150
<4> [177.725741] ? __pfx_kthread+0x10/0x10
<4> [177.725744] ret_from_fork+0x3d4/0x480
<4> [177.725746] ? __pfx_kthread+0x10/0x10
<4> [177.725749] ret_from_fork_asm+0x1a/0x30
<4> [177.725757] </TASK>
<4> [177.725758] irq event stamp: 487013
<4> [177.725759] hardirqs last enabled at (487019): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.725762] hardirqs last disabled at (487024): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.725764] softirqs last enabled at (486048): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.725767] softirqs last disabled at (486041): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.725769] ---[ end trace 0000000000000000 ]---
<5> [177.727470] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53694, lrc_seqno=53694, guc_id=0, flags=0x73 in no process [-1]
<7> [177.727473] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.727537] ------------[ cut here ]------------
<4> [177.727538] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.727540] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.727610] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.727685] efi_pstore nfnetlink autofs4
<4> [177.727690] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.727693] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.727694] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.727696] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.727701] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.727770] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.727771] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.727773] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.727775] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.727776] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.727777] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.727778] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.727780] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.727781] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.727783] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.727784] PKRU: 55555554
<4> [177.727785] Call Trace:
<4> [177.727787] <TASK>
<4> [177.727796] ? lock_sync+0x100/0x100
<4> [177.727801] ? lock_release+0xd0/0x2b0
<4> [177.727807] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.727813] process_one_work+0x239/0x760
<4> [177.727819] worker_thread+0x200/0x3f0
<4> [177.727821] ? __pfx_worker_thread+0x10/0x10
<4> [177.727824] kthread+0x10d/0x150
<4> [177.727826] ? __pfx_kthread+0x10/0x10
<4> [177.727830] ret_from_fork+0x3d4/0x480
<4> [177.727832] ? __pfx_kthread+0x10/0x10
<4> [177.727836] ret_from_fork_asm+0x1a/0x30
<4> [177.727843] </TASK>
<4> [177.727844] irq event stamp: 489787
<4> [177.727845] hardirqs last enabled at (489793): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.727848] hardirqs last disabled at (489798): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.727850] softirqs last enabled at (489714): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.727853] softirqs last disabled at (489709): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.727855] ---[ end trace 0000000000000000 ]---
<7> [177.727870] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.727974] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [177.728073] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.728144] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.728150] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.728210] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.728420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.728515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.728611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.728704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.728801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.728888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.728977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.729065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.729151] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.729249] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.729367] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.730382] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.740797] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.741045] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.742055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.742135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.742212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.742290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.742367] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.742441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.742516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.742590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.742667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.742743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.742827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.742902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.742975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.743049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.743122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.743196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.743269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.743343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.743417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.743498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.743578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.743657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.743737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.743825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.743904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.743982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.744058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.744138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.744217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.744295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.744373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.744452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.744535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.744618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.744701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.744782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.744881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.744963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.745042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.745118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.745199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.745277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.745354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.745430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.745515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.745595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.745674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.745752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.745841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.745921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.745998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.746077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.746155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.746232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.746308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.746385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.746460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.746535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.746613] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.746725] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.746729] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53694, lrc_seqno=53694, guc_id=0, flags=0x73 in no process [-1]
<7> [177.746732] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.747832] ------------[ cut here ]------------
<4> [177.747836] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.747841] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:9/209
<4> [177.747983] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.748072] efi_pstore nfnetlink autofs4
<4> [177.748080] CPU: 6 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.748085] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.748087] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.748089] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [177.748039] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [177.748098] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.748215] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.748217] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.748221] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.748223] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.748225] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.748226] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.748229] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.748231] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [177.748234] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.748236] CR2: 000077a5ba55f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.748239] PKRU: 55555554
<4> [177.748241] Call Trace:
<4> [177.748243] <TASK>
<4> [177.748251] ? lock_sync+0x100/0x100
<7> [177.748190] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [177.748259] ? lock_release+0xd0/0x2b0
<4> [177.748268] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.748277] process_one_work+0x239/0x760
<4> [177.748286] worker_thread+0x200/0x3f0
<4> [177.748290] ? __pfx_worker_thread+0x10/0x10
<4> [177.748293] kthread+0x10d/0x150
<4> [177.748296] ? __pfx_kthread+0x10/0x10
<4> [177.748302] ret_from_fork+0x3d4/0x480
<4> [177.748305] ? __pfx_kthread+0x10/0x10
<4> [177.748310] ret_from_fork_asm+0x1a/0x30
<4> [177.748322] </TASK>
<4> [177.748324] irq event stamp: 2407441
<4> [177.748325] hardirqs last enabled at (2407447): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.748329] hardirqs last disabled at (2407452): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.748332] softirqs last enabled at (2406340): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.748335] softirqs last disabled at (2406333): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.748339] ---[ end trace 0000000000000000 ]---
<6> [177.748342] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.748416] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.748424] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.748964] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.749170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.749260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.749355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.749443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.749531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.749616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.749701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.749795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.749878] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.749972] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.750092] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.751103] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.761812] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.762062] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.763023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.763105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.763184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.763260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.763336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.763411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.763485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.763557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.763631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.763704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.763780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.763873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.763952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.764030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.764106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.764181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.764255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.764331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.764409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.764491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.764573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.764656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.764740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.764830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.764908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.764985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.765066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.765148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.765230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.765308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.765387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.765466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.765550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.765634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.765719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.765810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.765893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.765972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.766051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.766127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.766206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.766283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.766360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.766436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.766516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.766594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.766674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.766753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.766842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.766919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.766996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.767076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.767154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.767232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.767308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.767387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.767463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.767539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.767618] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.767730] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.767734] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53694, lrc_seqno=53694, guc_id=0, flags=0x73 in no process [-1]
<7> [177.767737] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.768654] ------------[ cut here ]------------
<4> [177.768656] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.768657] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:9/209
<4> [177.768736] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.768809] efi_pstore nfnetlink autofs4
<4> [177.768814] CPU: 6 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.768817] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.768818] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.768820] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.768825] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.768896] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.768897] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.768900] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.768901] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.768902] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.768904] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.768905] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.768906] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [177.768908] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.768909] CR2: 000077a5ba55f000 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.768911] PKRU: 55555554
<4> [177.768912] Call Trace:
<4> [177.768913] <TASK>
<4> [177.768917] ? lock_sync+0x100/0x100
<4> [177.768921] ? lock_release+0xd0/0x2b0
<4> [177.768927] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.768932] process_one_work+0x239/0x760
<4> [177.768939] worker_thread+0x200/0x3f0
<4> [177.768941] ? __pfx_worker_thread+0x10/0x10
<4> [177.768944] kthread+0x10d/0x150
<4> [177.768947] ? __pfx_kthread+0x10/0x10
<4> [177.768950] ret_from_fork+0x3d4/0x480
<4> [177.768952] ? __pfx_kthread+0x10/0x10
<4> [177.768956] ret_from_fork_asm+0x1a/0x30
<4> [177.768963] </TASK>
<4> [177.768964] irq event stamp: 2410549
<4> [177.768965] hardirqs last enabled at (2410555): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.768968] hardirqs last disabled at (2410560): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.768970] softirqs last enabled at (2410202): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.768973] softirqs last disabled at (2410195): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.768975] ---[ end trace 0000000000000000 ]---
<7> [177.769601] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.769706] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [177.769797] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53749, lrc_seqno=53749, guc_id=0, flags=0x73 in no process [-1]
<7> [177.769801] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.769867] ------------[ cut here ]------------
<4> [177.769869] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.769870] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.769942] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.770006] efi_pstore nfnetlink autofs4
<4> [177.770010] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.770013] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.770014] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.770016] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.770021] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.770092] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.770093] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.770096] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.770097] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.770098] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.770099] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.770101] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.770102] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.770104] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.770105] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.770106] PKRU: 55555554
<4> [177.770107] Call Trace:
<4> [177.770109] <TASK>
<4> [177.770113] ? lock_sync+0x100/0x100
<4> [177.770117] ? lock_release+0xd0/0x2b0
<4> [177.770122] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.770128] process_one_work+0x239/0x760
<4> [177.770134] worker_thread+0x200/0x3f0
<4> [177.770137] ? __pfx_worker_thread+0x10/0x10
<4> [177.770139] kthread+0x10d/0x150
<4> [177.770142] ? __pfx_kthread+0x10/0x10
<4> [177.770146] ret_from_fork+0x3d4/0x480
<4> [177.770147] ? __pfx_kthread+0x10/0x10
<4> [177.770151] ret_from_fork_asm+0x1a/0x30
<4> [177.770158] </TASK>
<4> [177.770159] irq event stamp: 2413339
<4> [177.770160] hardirqs last enabled at (2413345): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.770163] hardirqs last disabled at (2413350): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.770165] softirqs last enabled at (2410202): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.770168] softirqs last disabled at (2410195): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.770170] ---[ end trace 0000000000000000 ]---
<6> [177.770172] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.770243] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.770249] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.770297] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.770504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.770594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.770688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.770776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.770878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.770964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.771053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.771145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.771229] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.771326] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.771445] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.772454] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.782792] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.783041] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.784049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.784136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.784220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.784307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.784395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.784482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.784571] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.784661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.784748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.784842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.784925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.785009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.785092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.785174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.785255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.785333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.785411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.785489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.785568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.785656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.785740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.785828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.785910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.785990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.786068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.786146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.786223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.786304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.786385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.786465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.786548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.786632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.786719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.786807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.786893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.786974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.787056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.787134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.787215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.787294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.787374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.787455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.787533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.787611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.787691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.787768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.787859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.787940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.788021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.788099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.788176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.788257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.788334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.788411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.788488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.788565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.788641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.788717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.788801] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.788913] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.788917] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53749, lrc_seqno=53749, guc_id=0, flags=0x73 in no process [-1]
<7> [177.788920] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.788978] ------------[ cut here ]------------
<4> [177.788980] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.788981] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.789053] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.789117] efi_pstore nfnetlink autofs4
<4> [177.789121] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.789124] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.789125] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.789126] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.789131] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.789201] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.789203] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.789205] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.789207] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.789208] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.789209] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.789210] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.789212] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.789213] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.789214] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.789216] PKRU: 55555554
<4> [177.789217] Call Trace:
<4> [177.789218] <TASK>
<4> [177.789222] ? lock_sync+0x100/0x100
<4> [177.789226] ? lock_release+0xd0/0x2b0
<4> [177.789232] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.789238] process_one_work+0x239/0x760
<4> [177.789243] worker_thread+0x200/0x3f0
<4> [177.789246] ? __pfx_worker_thread+0x10/0x10
<4> [177.789249] kthread+0x10d/0x150
<4> [177.789251] ? __pfx_kthread+0x10/0x10
<4> [177.789255] ret_from_fork+0x3d4/0x480
<4> [177.789257] ? __pfx_kthread+0x10/0x10
<4> [177.789260] ret_from_fork_asm+0x1a/0x30
<4> [177.789268] </TASK>
<4> [177.789269] irq event stamp: 495309
<4> [177.789270] hardirqs last enabled at (495315): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.789273] hardirqs last disabled at (495320): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.789275] softirqs last enabled at (494388): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.789277] softirqs last disabled at (494383): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.789280] ---[ end trace 0000000000000000 ]---
<6> [177.789281] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.789351] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.789357] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.789760] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.790196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.790286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.790379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.790467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.790554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.790644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.790734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.790831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.790916] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.791012] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.791130] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.792136] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.803126] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.803375] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.804471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.804552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.804630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.804707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.804789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.804866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.804941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.805016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.805091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.805166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.805241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.805315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.805390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.805464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.805539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.805612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.805686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.805760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.805844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.805926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.806009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.806090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.806173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.806252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.806333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.806412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.806490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.806570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.806650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.806729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.806814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.806893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.806980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.807070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.807157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.807241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.807323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.807402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.807481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.807564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.807648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.807730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.807820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.807898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.807979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.808056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.808134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.808212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.808294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.808374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.808452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.808534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.808613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.808695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.808775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.808869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.808946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.809023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.809103] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.809214] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.809218] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53749, lrc_seqno=53749, guc_id=0, flags=0x73 in no process [-1]
<7> [177.809221] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.809280] ------------[ cut here ]------------
<4> [177.809281] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.809282] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.809353] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.809416] efi_pstore nfnetlink autofs4
<4> [177.809421] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.809423] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.809425] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.809426] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.809431] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.809501] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.809502] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.809504] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.809506] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.809507] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.809508] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.809510] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.809511] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.809513] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.809514] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.809515] PKRU: 55555554
<4> [177.809517] Call Trace:
<4> [177.809518] <TASK>
<4> [177.809522] ? lock_sync+0x100/0x100
<4> [177.809526] ? lock_release+0xd0/0x2b0
<4> [177.809531] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.809537] process_one_work+0x239/0x760
<4> [177.809543] worker_thread+0x200/0x3f0
<4> [177.809546] ? __pfx_worker_thread+0x10/0x10
<4> [177.809548] kthread+0x10d/0x150
<4> [177.809551] ? __pfx_kthread+0x10/0x10
<4> [177.809555] ret_from_fork+0x3d4/0x480
<4> [177.809557] ? __pfx_kthread+0x10/0x10
<4> [177.809560] ret_from_fork_asm+0x1a/0x30
<4> [177.809567] </TASK>
<4> [177.809568] irq event stamp: 498417
<4> [177.809570] hardirqs last enabled at (498423): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.809572] hardirqs last disabled at (498428): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.809575] softirqs last enabled at (497172): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.809577] softirqs last disabled at (497163): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.809579] ---[ end trace 0000000000000000 ]---
<7> [177.810795] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.810903] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [177.811392] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53804, lrc_seqno=53804, guc_id=0, flags=0x73 in no process [-1]
<7> [177.811397] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.811461] ------------[ cut here ]------------
<4> [177.811462] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.811464] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.811538] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.811603] efi_pstore nfnetlink autofs4
<4> [177.811607] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.811610] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.811611] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.811613] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.811618] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.811689] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.811691] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.811693] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.811694] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.811696] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.811697] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.811698] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.811699] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.811701] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.811702] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.811704] PKRU: 55555554
<4> [177.811705] Call Trace:
<4> [177.811706] <TASK>
<4> [177.811710] ? lock_sync+0x100/0x100
<4> [177.811715] ? lock_release+0xd0/0x2b0
<4> [177.811721] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.811726] process_one_work+0x239/0x760
<4> [177.811733] worker_thread+0x200/0x3f0
<4> [177.811736] ? __pfx_worker_thread+0x10/0x10
<4> [177.811738] kthread+0x10d/0x150
<4> [177.811741] ? __pfx_kthread+0x10/0x10
<4> [177.811744] ret_from_fork+0x3d4/0x480
<4> [177.811747] ? __pfx_kthread+0x10/0x10
<4> [177.811750] ret_from_fork_asm+0x1a/0x30
<4> [177.811757] </TASK>
<4> [177.811758] irq event stamp: 2418219
<4> [177.811760] hardirqs last enabled at (2418225): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.811763] hardirqs last disabled at (2418230): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.811765] softirqs last enabled at (2410202): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.811767] softirqs last disabled at (2410195): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.811770] ---[ end trace 0000000000000000 ]---
<6> [177.811772] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.811861] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.811867] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.811914] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.812120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.812213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.812308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.812398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.812485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.812570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.812657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.812746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.812845] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.812942] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.813062] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.814070] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.824787] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.825035] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.826062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.826142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.826220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.826299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.826375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.826451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.826527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.826602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.826677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.826753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.826838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.826918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.826995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.827071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.827147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.827222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.827297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.827372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.827446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.827532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.827617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.827697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.827777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.827875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.827954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.828032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.828110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.828190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.828270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.828351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.828434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.828515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.828600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.828685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.828773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.828870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.828952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.829031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.829109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.829186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.829267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.829349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.829431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.829509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.829594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.829675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.829754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.829838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.829916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.829992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.830068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.830147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.830225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.830307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.830386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.830465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.830543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.830620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.830700] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.830812] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.830816] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53804, lrc_seqno=53804, guc_id=0, flags=0x73 in no process [-1]
<7> [177.830819] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.830876] ------------[ cut here ]------------
<4> [177.830878] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.830879] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.830950] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.831013] efi_pstore nfnetlink autofs4
<4> [177.831017] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.831020] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.831021] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.831022] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.831027] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.831100] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.831101] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.831103] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.831105] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.831106] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.831107] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.831108] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.831110] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.831111] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.831113] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.831114] PKRU: 55555554
<4> [177.831115] Call Trace:
<4> [177.831117] <TASK>
<4> [177.831120] ? lock_sync+0x100/0x100
<4> [177.831125] ? lock_release+0xd0/0x2b0
<4> [177.831130] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.831135] process_one_work+0x239/0x760
<4> [177.831141] worker_thread+0x200/0x3f0
<4> [177.831144] ? __pfx_worker_thread+0x10/0x10
<4> [177.831146] kthread+0x10d/0x150
<4> [177.831149] ? __pfx_kthread+0x10/0x10
<4> [177.831153] ret_from_fork+0x3d4/0x480
<4> [177.831155] ? __pfx_kthread+0x10/0x10
<4> [177.831158] ret_from_fork_asm+0x1a/0x30
<4> [177.831165] </TASK>
<4> [177.831167] irq event stamp: 503807
<4> [177.831168] hardirqs last enabled at (503813): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.831171] hardirqs last disabled at (503818): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.831173] softirqs last enabled at (502892): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.831175] softirqs last disabled at (502887): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.831177] ---[ end trace 0000000000000000 ]---
<6> [177.831179] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.831252] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.831258] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.831663] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.831871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.831960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.832053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.832142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.832230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.832316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.832404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.832493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.832576] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.832672] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.832796] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.833805] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.843785] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.844034] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.845194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.845273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.845350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.845428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.845505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.845584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.845662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.845738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.845824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.845900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.845974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.846049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.846124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.846198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.846273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.846347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.846422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.846496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.846570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.846653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.846737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.846826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.846908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.846987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.847065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.847143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.847220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.847299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.847381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.847460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.847540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.847619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.847705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.847799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.847887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.847972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.848054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.848134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.848213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.848289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.848370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.848447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.848524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.848600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.848680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.848757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.848844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.848920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.848998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.849075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.849151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.849230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.849307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.849385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.849463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.849547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.849626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.849704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.849787] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.849897] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.849901] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53804, lrc_seqno=53804, guc_id=0, flags=0x73 in no process [-1]
<7> [177.849903] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.849961] ------------[ cut here ]------------
<4> [177.849962] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.849964] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.850035] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.850099] efi_pstore nfnetlink autofs4
<4> [177.850103] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.850106] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.850107] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.850108] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.850113] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.850183] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.850185] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.850187] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.850189] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.850190] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.850191] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.850192] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.850194] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.850195] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.850197] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.850198] PKRU: 55555554
<4> [177.850199] Call Trace:
<4> [177.850200] <TASK>
<4> [177.850204] ? lock_sync+0x100/0x100
<4> [177.850209] ? lock_release+0xd0/0x2b0
<4> [177.850214] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.850220] process_one_work+0x239/0x760
<4> [177.850226] worker_thread+0x200/0x3f0
<4> [177.850228] ? __pfx_worker_thread+0x10/0x10
<4> [177.850231] kthread+0x10d/0x150
<4> [177.850234] ? __pfx_kthread+0x10/0x10
<4> [177.850237] ret_from_fork+0x3d4/0x480
<4> [177.850239] ? __pfx_kthread+0x10/0x10
<4> [177.850242] ret_from_fork_asm+0x1a/0x30
<4> [177.850250] </TASK>
<4> [177.850251] irq event stamp: 506903
<4> [177.850252] hardirqs last enabled at (506909): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.850255] hardirqs last disabled at (506914): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.850257] softirqs last enabled at (505982): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.850259] softirqs last disabled at (505973): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.850261] ---[ end trace 0000000000000000 ]---
<5> [177.851882] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53859, lrc_seqno=53859, guc_id=0, flags=0x73 in no process [-1]
<7> [177.851887] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [177.851892] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [177.851979] ------------[ cut here ]------------
<4> [177.851980] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.851982] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<7> [177.851995] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [177.852088] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.852216] efi_pstore nfnetlink autofs4
<4> [177.852224] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.852228] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.852231] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.852233] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.852242] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.852347] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.852350] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.852354] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.852357] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.852359] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.852362] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.852364] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.852366] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.852369] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.852372] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.852375] PKRU: 55555554
<4> [177.852377] Call Trace:
<4> [177.852379] <TASK>
<4> [177.852386] ? lock_sync+0x100/0x100
<4> [177.852394] ? lock_release+0xd0/0x2b0
<4> [177.852404] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.852414] process_one_work+0x239/0x760
<4> [177.852425] worker_thread+0x200/0x3f0
<4> [177.852430] ? __pfx_worker_thread+0x10/0x10
<4> [177.852434] kthread+0x10d/0x150
<4> [177.852439] ? __pfx_kthread+0x10/0x10
<4> [177.852445] ret_from_fork+0x3d4/0x480
<4> [177.852448] ? __pfx_kthread+0x10/0x10
<4> [177.852454] ret_from_fork_asm+0x1a/0x30
<4> [177.852468] </TASK>
<4> [177.852470] irq event stamp: 2423323
<4> [177.852472] hardirqs last enabled at (2423329): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.852476] hardirqs last disabled at (2423334): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.852480] softirqs last enabled at (2419310): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.852484] softirqs last disabled at (2419303): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.852488] ---[ end trace 0000000000000000 ]---
<6> [177.852491] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.852596] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.852605] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.852692] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.852945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.853072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.853207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.853337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.853463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.853586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.853710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.853857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.853980] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.854115] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.854274] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.855409] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.865796] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.866105] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.867266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.867383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.867498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.867612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.867727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.867863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.867974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.868088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.868199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.868310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.868423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.868536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.868648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.868759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.868902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.869011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.869124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.869233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.869345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.869466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.869586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.869703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.869833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.869948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.870065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.870180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.870293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.870410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.870528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.870644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.870762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.870911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.871038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.871162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.871287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.871411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.871530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.871646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.871762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.871866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.871951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.872035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.872117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.872195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.872278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.872357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.872435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.872513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.872591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.872667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.872744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.872835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.872912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.872993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.873074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.873154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.873232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.873309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.873388] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.873500] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.873505] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53859, lrc_seqno=53859, guc_id=0, flags=0x73 in no process [-1]
<7> [177.873507] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.873565] ------------[ cut here ]------------
<4> [177.873566] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.873568] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.873639] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.873702] efi_pstore nfnetlink autofs4
<4> [177.873706] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.873709] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.873710] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.873712] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.873717] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.873792] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.873794] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.873797] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.873799] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.873800] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.873801] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.873803] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.873804] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.873806] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.873808] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.873809] PKRU: 55555554
<4> [177.873810] Call Trace:
<4> [177.873811] <TASK>
<4> [177.873815] ? lock_sync+0x100/0x100
<4> [177.873820] ? lock_release+0xd0/0x2b0
<4> [177.873826] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.873832] process_one_work+0x239/0x760
<4> [177.873838] worker_thread+0x200/0x3f0
<4> [177.873841] ? __pfx_worker_thread+0x10/0x10
<4> [177.873843] kthread+0x10d/0x150
<4> [177.873846] ? __pfx_kthread+0x10/0x10
<4> [177.873850] ret_from_fork+0x3d4/0x480
<4> [177.873852] ? __pfx_kthread+0x10/0x10
<4> [177.873855] ret_from_fork_asm+0x1a/0x30
<4> [177.873863] </TASK>
<4> [177.873864] irq event stamp: 2426417
<4> [177.873865] hardirqs last enabled at (2426423): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.873868] hardirqs last disabled at (2426428): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.873870] softirqs last enabled at (2425526): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.873873] softirqs last disabled at (2425521): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.873875] ---[ end trace 0000000000000000 ]---
<6> [177.873877] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.873947] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.873953] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.875192] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.875296] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.875421] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.875630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.875722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.875825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.875915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.876001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.876087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.876174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.876262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.876344] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.876440] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.876561] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.877577] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.887779] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.888333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.889356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.889437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.889514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.889592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.889669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.889745] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.889829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.889904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.889978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.890055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.890133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.890209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.890285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.890360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.890434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.890509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.890582] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.890656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.890729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.890819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.890900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.890979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.891060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.891140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.891221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.891300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.891379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.891458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.891537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.891615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.891693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.891775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.891860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.891944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.892028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.892109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.892187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.892263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.892342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.892423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.892505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.892585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.892663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.892739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.892833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.892911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.892987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.893063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.893144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.893223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.893302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.893383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.893461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.893539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.893616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.893694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.893774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.893851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.893930] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.894042] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.894046] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53859, lrc_seqno=53859, guc_id=0, flags=0x73 in no process [-1]
<7> [177.894049] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.894108] ------------[ cut here ]------------
<4> [177.894109] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.894110] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [177.894183] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.894249] efi_pstore nfnetlink autofs4
<4> [177.894254] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.894257] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.894258] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.894259] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.894264] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.894334] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.894336] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.894338] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.894340] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.894341] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.894342] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.894343] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.894345] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [177.894346] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.894348] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [177.894349] PKRU: 55555554
<4> [177.894350] Call Trace:
<4> [177.894351] <TASK>
<4> [177.894356] ? lock_sync+0x100/0x100
<4> [177.894361] ? lock_release+0xd0/0x2b0
<4> [177.894366] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.894372] process_one_work+0x239/0x760
<4> [177.894378] worker_thread+0x200/0x3f0
<4> [177.894381] ? __pfx_worker_thread+0x10/0x10
<4> [177.894383] kthread+0x10d/0x150
<4> [177.894386] ? __pfx_kthread+0x10/0x10
<4> [177.894390] ret_from_fork+0x3d4/0x480
<4> [177.894392] ? __pfx_kthread+0x10/0x10
<4> [177.894395] ret_from_fork_asm+0x1a/0x30
<4> [177.894403] </TASK>
<4> [177.894404] irq event stamp: 119313
<4> [177.894405] hardirqs last enabled at (119319): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.894408] hardirqs last disabled at (119324): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.894410] softirqs last enabled at (118376): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.894412] softirqs last disabled at (118349): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.894415] ---[ end trace 0000000000000000 ]---
<5> [177.895785] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53914, lrc_seqno=53914, guc_id=0, flags=0x73 in no process [-1]
<7> [177.895794] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.895903] ------------[ cut here ]------------
<4> [177.895904] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.895906] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [177.895981] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.896046] efi_pstore
<7> [177.895996] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [177.896048] nfnetlink autofs4
<4> [177.896051] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.896054] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.896055] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.896056] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.896062] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.896135] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.896137] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.896139] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.896141] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.896142] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.896143] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.896144] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.896146] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [177.896147] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.896149] CR2: 000060dda491f42c CR3: 000000012cae9001 CR4: 0000000000f72ef0
<4> [177.896150] PKRU: 55555554
<4> [177.896151] Call Trace:
<7> [177.896103] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [177.896153] <TASK>
<4> [177.896156] ? lock_sync+0x100/0x100
<4> [177.896161] ? lock_release+0xd0/0x2b0
<4> [177.896166] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.896171] process_one_work+0x239/0x760
<4> [177.896177] worker_thread+0x200/0x3f0
<4> [177.896180] ? __pfx_worker_thread+0x10/0x10
<4> [177.896182] kthread+0x10d/0x150
<4> [177.896185] ? __pfx_kthread+0x10/0x10
<4> [177.896188] ret_from_fork+0x3d4/0x480
<4> [177.896190] ? __pfx_kthread+0x10/0x10
<4> [177.896193] ret_from_fork_asm+0x1a/0x30
<4> [177.896200] </TASK>
<4> [177.896201] irq event stamp: 122183
<4> [177.896202] hardirqs last enabled at (122189): [<ffffffff814b0064>] vprintk_emit+0x414/0x470
<4> [177.896205] hardirqs last disabled at (122194): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.896207] softirqs last enabled at (120382): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.896210] softirqs last disabled at (120375): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.896212] ---[ end trace 0000000000000000 ]---
<6> [177.896214] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.896286] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.896293] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.896341] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.896544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.896635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.896728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.896830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.896919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.897005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.897092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.897180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.897265] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.897360] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.897477] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.898486] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.908777] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.909027] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.910014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.910093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.910169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.910246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.910324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.910399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.910478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.910556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.910632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.910707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.910794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.910869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.910944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.911018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.911092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.911167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.911240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.911314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.911387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.911468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.911551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.911633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.911715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.911802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.911880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.911958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.912034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.912114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.912194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.912273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.912351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.912430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.912515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.912598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.912681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.912762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.912855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.912933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.913010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.913085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.913164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.913242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.913320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.913396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.913476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.913553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.913633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.913714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.913803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.913881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.913958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.914038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.914115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.914193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.914269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.914348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.914428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.914508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.914589] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.914701] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.914706] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53914, lrc_seqno=53914, guc_id=0, flags=0x73 in no process [-1]
<7> [177.914708] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.914767] ------------[ cut here ]------------
<4> [177.914772] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.914773] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [177.914844] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.914907] efi_pstore nfnetlink autofs4
<4> [177.914911] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.914914] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.914915] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.914917] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.914922] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.914992] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.914994] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.914996] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.914997] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.914998] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.915000] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.915001] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.915002] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [177.915004] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.915005] CR2: 000060dda491f42c CR3: 0000000114e1a004 CR4: 0000000000f72ef0
<4> [177.915007] PKRU: 55555554
<4> [177.915008] Call Trace:
<4> [177.915009] <TASK>
<4> [177.915013] ? lock_sync+0x100/0x100
<4> [177.915017] ? lock_release+0xd0/0x2b0
<4> [177.915022] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.915028] process_one_work+0x239/0x760
<4> [177.915034] worker_thread+0x200/0x3f0
<4> [177.915037] ? __pfx_worker_thread+0x10/0x10
<4> [177.915039] kthread+0x10d/0x150
<4> [177.915042] ? __pfx_kthread+0x10/0x10
<4> [177.915046] ret_from_fork+0x3d4/0x480
<4> [177.915048] ? __pfx_kthread+0x10/0x10
<4> [177.915051] ret_from_fork_asm+0x1a/0x30
<4> [177.915058] </TASK>
<4> [177.915059] irq event stamp: 125323
<4> [177.915061] hardirqs last enabled at (125329): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.915063] hardirqs last disabled at (125334): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.915066] softirqs last enabled at (124414): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.915068] softirqs last disabled at (124391): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.915070] ---[ end trace 0000000000000000 ]---
<6> [177.915072] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.915145] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.915150] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.916399] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.916505] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.916613] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.916826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.916919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.917014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.917105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.917195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.917283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.917370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.917458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.917541] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.917638] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.917754] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.918774] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.929501] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.929749] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.930750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.930841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.930919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.930996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.931071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.931146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.931220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.931295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.931369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.931444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.931517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.931591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.931665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.931739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.931823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.931897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.931971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.932045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.932118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.932200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.932281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.932361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.932444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.932526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.932606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.932685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.932763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.932870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.932950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.933029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.933108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.933190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.933279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.933365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.933452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.933535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.933616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.933695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.933775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.933852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.933932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.934010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.934087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.934168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.934252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.934332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.934412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.934490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.934568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.934645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.934722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.934809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.934889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.934972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.935051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.935131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.935208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.935285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.935365] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.935476] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.935480] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53914, lrc_seqno=53914, guc_id=0, flags=0x73 in no process [-1]
<7> [177.935482] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.935541] ------------[ cut here ]------------
<4> [177.935542] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.935543] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [177.935615] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.935680] efi_pstore nfnetlink autofs4
<4> [177.935684] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.935687] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.935688] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.935690] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.935695] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.935765] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.935770] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [177.935772] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.935774] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.935775] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.935776] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.935778] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.935779] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [177.935781] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.935782] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.935784] PKRU: 55555554
<4> [177.935785] Call Trace:
<4> [177.935787] <TASK>
<4> [177.935791] ? lock_sync+0x100/0x100
<4> [177.935795] ? lock_release+0xd0/0x2b0
<4> [177.935801] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.935806] process_one_work+0x239/0x760
<4> [177.935813] worker_thread+0x200/0x3f0
<4> [177.935816] ? __pfx_worker_thread+0x10/0x10
<4> [177.935818] kthread+0x10d/0x150
<4> [177.935821] ? __pfx_kthread+0x10/0x10
<4> [177.935825] ret_from_fork+0x3d4/0x480
<4> [177.935827] ? __pfx_kthread+0x10/0x10
<4> [177.935830] ret_from_fork_asm+0x1a/0x30
<4> [177.935838] </TASK>
<4> [177.935839] irq event stamp: 512941
<4> [177.935840] hardirqs last enabled at (512947): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.935843] hardirqs last disabled at (512952): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.935845] softirqs last enabled at (511962): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.935848] softirqs last disabled at (511955): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.935850] ---[ end trace 0000000000000000 ]---
<5> [177.936453] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53969, lrc_seqno=53969, guc_id=0, flags=0x73 in no process [-1]
<7> [177.936461] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.936729] ------------[ cut here ]------------
<4> [177.936731] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.936734] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:39/3057
<4> [177.936863] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.936978] efi_pstore nfnetlink autofs4
<4> [177.936987] CPU: 12 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.936993] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.936996] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.936999] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.937010] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.937126] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.937130] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.937134] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.937136] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.937138] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.937140] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.937142] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.937144] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [177.937147] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.937150] CR2: 0000768ff00973c8 CR3: 0000000115508001 CR4: 0000000000f72ef0
<4> [177.937153] PKRU: 55555554
<4> [177.937154] Call Trace:
<4> [177.937156] <TASK>
<4> [177.937164] ? lock_sync+0x100/0x100
<4> [177.937172] ? lock_release+0xd0/0x2b0
<4> [177.937182] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.937192] process_one_work+0x239/0x760
<4> [177.937203] worker_thread+0x200/0x3f0
<4> [177.937207] ? __pfx_worker_thread+0x10/0x10
<4> [177.937211] kthread+0x10d/0x150
<4> [177.937215] ? __pfx_kthread+0x10/0x10
<4> [177.937221] ret_from_fork+0x3d4/0x480
<4> [177.937224] ? __pfx_kthread+0x10/0x10
<4> [177.937229] ret_from_fork_asm+0x1a/0x30
<4> [177.937242] </TASK>
<4> [177.937243] irq event stamp: 127547
<4> [177.937245] hardirqs last enabled at (127553): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.937250] hardirqs last disabled at (127558): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.937253] softirqs last enabled at (125464): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [177.937258] softirqs last disabled at (125462): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [177.937262] ---[ end trace 0000000000000000 ]---
<6> [177.937265] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.937383] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.937392] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.937444] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.937550] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.937662] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.937923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.938059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.938198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.938332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.938465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.938597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.938730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.938870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.939001] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.939144] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.939311] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.940449] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.950773] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.951026] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.952044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.952125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.952203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.952282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.952359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.952436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.952515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.952594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.952672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.952747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.952838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.952913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.952988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.953063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.953137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.953212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.953287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.953362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.953440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.953523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.953605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.953685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.953769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.953847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.953924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.954001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.954078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.954162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.954246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.954327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.954407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.954487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.954575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.954663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.954750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.954847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.954929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.955010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.955088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.955165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.955246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.955324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.955401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.955477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.955558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.955633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.955711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.955801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.955884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.955963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.956040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.956119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.956196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.956273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.956349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.956427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.956503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.956579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.956658] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.956779] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.956784] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53969, lrc_seqno=53969, guc_id=0, flags=0x73 in no process [-1]
<7> [177.956786] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.956846] ------------[ cut here ]------------
<4> [177.956847] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.956849] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [177.956922] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.956987] efi_pstore nfnetlink autofs4
<4> [177.956991] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.956994] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.956996] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.956997] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.957002] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.957073] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.957075] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [177.957077] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.957079] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.957080] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.957081] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.957082] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.957084] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [177.957085] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.957087] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [177.957088] PKRU: 55555554
<4> [177.957089] Call Trace:
<4> [177.957090] <TASK>
<4> [177.957094] ? lock_sync+0x100/0x100
<4> [177.957099] ? lock_release+0xd0/0x2b0
<4> [177.957104] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.957110] process_one_work+0x239/0x760
<4> [177.957116] worker_thread+0x200/0x3f0
<4> [177.957119] ? __pfx_worker_thread+0x10/0x10
<4> [177.957121] kthread+0x10d/0x150
<4> [177.957124] ? __pfx_kthread+0x10/0x10
<4> [177.957128] ret_from_fork+0x3d4/0x480
<4> [177.957130] ? __pfx_kthread+0x10/0x10
<4> [177.957133] ret_from_fork_asm+0x1a/0x30
<4> [177.957140] </TASK>
<4> [177.957142] irq event stamp: 130693
<4> [177.957143] hardirqs last enabled at (130699): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.957146] hardirqs last disabled at (130704): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.957148] softirqs last enabled at (129896): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.957150] softirqs last disabled at (129889): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.957153] ---[ end trace 0000000000000000 ]---
<6> [177.957155] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.957224] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.957232] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.958481] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.958587] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.958696] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.958913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.959004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.959098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.959188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.959276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.959362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.959447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.959534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.959616] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.959710] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.959840] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.960883] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.971609] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.971869] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.972895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.972976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.973053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.973133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.973211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.973288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.973362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.973436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.973512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.973587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.973662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.973737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.973823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.973912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.973986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.974060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.974136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.974214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.974291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.974374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.974456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.974535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.974616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.974694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.974777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.974881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.974963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.975045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.975128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.975208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.975288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.975369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.975453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.975539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.975624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.975707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.975798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.975891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.975968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.976046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.976127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.976205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.976284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.976365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.976451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.976532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.976611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.976689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.976783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.976874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.976950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.977030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.977108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.977186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.977263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.977342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.977420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.977500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.977583] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.977700] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.977704] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=53969, lrc_seqno=53969, guc_id=0, flags=0x73 in no process [-1]
<7> [177.977706] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.977774] ------------[ cut here ]------------
<4> [177.977775] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.977777] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.977849] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.977914] efi_pstore nfnetlink autofs4
<4> [177.977919] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.977922] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.977923] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.977925] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.977930] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.977999] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.978001] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.978003] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.978005] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.978006] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.978007] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.978008] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.978010] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.978011] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.978012] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.978014] PKRU: 55555554
<4> [177.978015] Call Trace:
<4> [177.978016] <TASK>
<4> [177.978020] ? lock_sync+0x100/0x100
<4> [177.978025] ? lock_release+0xd0/0x2b0
<4> [177.978030] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.978036] process_one_work+0x239/0x760
<4> [177.978042] worker_thread+0x200/0x3f0
<4> [177.978045] ? __pfx_worker_thread+0x10/0x10
<4> [177.978047] kthread+0x10d/0x150
<4> [177.978050] ? __pfx_kthread+0x10/0x10
<4> [177.978054] ret_from_fork+0x3d4/0x480
<4> [177.978056] ? __pfx_kthread+0x10/0x10
<4> [177.978059] ret_from_fork_asm+0x1a/0x30
<4> [177.978066] </TASK>
<4> [177.978067] irq event stamp: 2432315
<4> [177.978068] hardirqs last enabled at (2432321): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.978071] hardirqs last disabled at (2432326): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.978074] softirqs last enabled at (2431406): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.978076] softirqs last disabled at (2431401): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.978078] ---[ end trace 0000000000000000 ]---
<5> [177.978678] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54024, lrc_seqno=54024, guc_id=0, flags=0x73 in no process [-1]
<7> [177.978680] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.978740] ------------[ cut here ]------------
<4> [177.978741] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.978742] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:9/209
<4> [177.978824] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.978888] efi_pstore nfnetlink autofs4
<4> [177.978892] CPU: 7 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.978894] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.978895] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.978897] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.978901] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.978970] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.978972] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.978974] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.978975] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.978976] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.978977] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.978979] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.978980] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000
<4> [177.978981] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.978983] CR2: 0000758009e5e558 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [177.978984] PKRU: 55555554
<4> [177.978985] Call Trace:
<4> [177.978986] <TASK>
<4> [177.978990] ? lock_sync+0x100/0x100
<4> [177.978994] ? lock_release+0xd0/0x2b0
<4> [177.979000] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.979005] process_one_work+0x239/0x760
<4> [177.979011] worker_thread+0x200/0x3f0
<4> [177.979014] ? __pfx_worker_thread+0x10/0x10
<4> [177.979016] kthread+0x10d/0x150
<4> [177.979019] ? __pfx_kthread+0x10/0x10
<4> [177.979022] ret_from_fork+0x3d4/0x480
<4> [177.979024] ? __pfx_kthread+0x10/0x10
<4> [177.979027] ret_from_fork_asm+0x1a/0x30
<4> [177.979035] </TASK>
<4> [177.979036] irq event stamp: 2435057
<4> [177.979037] hardirqs last enabled at (2435063): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.979039] hardirqs last disabled at (2435068): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.979041] softirqs last enabled at (2431406): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.979044] softirqs last disabled at (2431401): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.979046] ---[ end trace 0000000000000000 ]---
<6> [177.979782] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.979921] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.979929] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [177.979963] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [177.980072] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [177.980178] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [177.980383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [177.980471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [177.980565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [177.980656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [177.980745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [177.980846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [177.980933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [177.981022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [177.981105] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [177.981203] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [177.981323] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [177.982333] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [177.992769] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [177.993018] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [177.994060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [177.994139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [177.994218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [177.994298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [177.994377] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [177.994454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [177.994530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [177.994605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [177.994680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [177.994755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [177.994858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [177.994932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [177.995007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [177.995086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [177.995163] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [177.995238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [177.995313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [177.995387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [177.995461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [177.995543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [177.995626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [177.995705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [177.995795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [177.995877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [177.995957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [177.996035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [177.996112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [177.996191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [177.996272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [177.996355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [177.996437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [177.996520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [177.996605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [177.996689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [177.996779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [177.996860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [177.996940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [177.997021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [177.997101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [177.997179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [177.997261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [177.997341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [177.997419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [177.997496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [177.997578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [177.997656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [177.997733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [177.997821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [177.997900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [177.997981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [177.998061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [177.998143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [177.998223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [177.998302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [177.998378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [177.998457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [177.998535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [177.998614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [177.998697] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [177.999179] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [177.999183] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54024, lrc_seqno=54024, guc_id=0, flags=0x73 in no process [-1]
<7> [177.999186] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [177.999246] ------------[ cut here ]------------
<4> [177.999248] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [177.999249] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:9/209
<4> [177.999321] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [177.999387] efi_pstore nfnetlink autofs4
<4> [177.999391] CPU: 6 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [177.999394] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [177.999395] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [177.999397] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [177.999403] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [177.999474] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [177.999475] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [177.999478] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [177.999480] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [177.999481] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [177.999483] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [177.999484] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [177.999486] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [177.999488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [177.999489] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [177.999491] PKRU: 55555554
<4> [177.999492] Call Trace:
<4> [177.999494] <TASK>
<4> [177.999498] ? lock_sync+0x100/0x100
<4> [177.999502] ? lock_release+0xd0/0x2b0
<4> [177.999508] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [177.999514] process_one_work+0x239/0x760
<4> [177.999521] worker_thread+0x200/0x3f0
<4> [177.999523] ? __pfx_worker_thread+0x10/0x10
<4> [177.999526] kthread+0x10d/0x150
<4> [177.999529] ? __pfx_kthread+0x10/0x10
<4> [177.999533] ret_from_fork+0x3d4/0x480
<4> [177.999535] ? __pfx_kthread+0x10/0x10
<4> [177.999539] ret_from_fork_asm+0x1a/0x30
<4> [177.999546] </TASK>
<4> [177.999548] irq event stamp: 2438209
<4> [177.999549] hardirqs last enabled at (2438215): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [177.999552] hardirqs last disabled at (2438220): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [177.999555] softirqs last enabled at (2437106): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.999558] softirqs last disabled at (2437099): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [177.999560] ---[ end trace 0000000000000000 ]---
<6> [177.999562] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [177.999634] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [177.999640] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.000891] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.000995] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.001100] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.001320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.001414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.001510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.001601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.001690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.001837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.001924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.002012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.002094] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.002191] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.002307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.003320] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.013768] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.014020] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.014994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.015074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.015151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.015229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.015306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.015383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.015460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.015536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.015612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.015690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.015773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.015850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.015926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.016002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.016077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.016151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.016226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.016299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.016372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.016453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.016535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.016614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.016696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.016783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.016865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.016944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.017022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.017102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.017183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.017262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.017345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.017426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.017511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.017595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.017679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.017764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.017844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.017922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.017999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.018075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.018153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.018231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.018308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.018384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.018464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.018545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.018626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.018703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.018788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.018864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.018940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.019020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.019102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.019182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.019259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.019338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.019414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.019490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.019569] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.019681] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.019685] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54024, lrc_seqno=54024, guc_id=0, flags=0x73 in no process [-1]
<7> [178.019688] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.019746] ------------[ cut here ]------------
<4> [178.019747] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.019749] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.019833] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.019900] efi_pstore nfnetlink autofs4
<4> [178.019905] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.019908] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.019909] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.019911] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.019916] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.019985] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.019987] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.019989] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.019991] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.019992] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.019993] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.019995] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.019996] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.019998] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.019999] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.020001] PKRU: 55555554
<4> [178.020002] Call Trace:
<4> [178.020003] <TASK>
<4> [178.020007] ? lock_sync+0x100/0x100
<4> [178.020012] ? lock_release+0xd0/0x2b0
<4> [178.020017] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.020023] process_one_work+0x239/0x760
<4> [178.020029] worker_thread+0x200/0x3f0
<4> [178.020032] ? __pfx_worker_thread+0x10/0x10
<4> [178.020034] kthread+0x10d/0x150
<4> [178.020037] ? __pfx_kthread+0x10/0x10
<4> [178.020041] ret_from_fork+0x3d4/0x480
<4> [178.020043] ? __pfx_kthread+0x10/0x10
<4> [178.020046] ret_from_fork_asm+0x1a/0x30
<4> [178.020054] </TASK>
<4> [178.020055] irq event stamp: 521259
<4> [178.020056] hardirqs last enabled at (521265): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.020059] hardirqs last disabled at (521270): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.020061] softirqs last enabled at (520350): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.020064] softirqs last disabled at (520343): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.020066] ---[ end trace 0000000000000000 ]---
<5> [178.021508] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54079, lrc_seqno=54079, guc_id=0, flags=0x73 in no process [-1]
<7> [178.021511] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.021572] ------------[ cut here ]------------
<4> [178.021574] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.021575] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.021647] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.021712] efi_pstore nfnetlink autofs4
<4> [178.021716] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.021718] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.021720] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.021721] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.021726] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.021804] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.021806] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.021808] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.021810] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.021811] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.021812] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.021814] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.021815] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.021817] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.021818] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.021819] PKRU: 55555554
<4> [178.021821] Call Trace:
<4> [178.021822] <TASK>
<4> [178.021825] ? lock_sync+0x100/0x100
<4> [178.021830] ? lock_release+0xd0/0x2b0
<4> [178.021835] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.021841] process_one_work+0x239/0x760
<4> [178.021847] worker_thread+0x200/0x3f0
<4> [178.021850] ? __pfx_worker_thread+0x10/0x10
<4> [178.021852] kthread+0x10d/0x150
<4> [178.021856] ? __pfx_kthread+0x10/0x10
<4> [178.021860] ret_from_fork+0x3d4/0x480
<4> [178.021862] ? __pfx_kthread+0x10/0x10
<4> [178.021865] ret_from_fork_asm+0x1a/0x30
<4> [178.021873] </TASK>
<4> [178.021874] irq event stamp: 523989
<4> [178.021875] hardirqs last enabled at (523995): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.021878] hardirqs last disabled at (524000): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.021880] softirqs last enabled at (520350): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.021882] softirqs last disabled at (520343): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.021885] ---[ end trace 0000000000000000 ]---
<7> [178.021899] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.022005] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.022104] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.022176] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.022182] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.022228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.022430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.022517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.022611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.022703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.022800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.022889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.022976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.023069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.023155] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.023251] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.023367] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.024375] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.034764] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.035015] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.036037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.036117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.036194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.036273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.036350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.036425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.036502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.036578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.036654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.036732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.036816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.036892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.036968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.037043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.037117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.037192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.037266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.037340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.037413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.037495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.037576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.037656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.037739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.037830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.037908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.037987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.038065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.038148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.038232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.038313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.038394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.038472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.038557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.038642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.038727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.038821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.038904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.038986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.039065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.039142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.039222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.039301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.039379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.039455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.039536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.039616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.039698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.039784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.039865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.039944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.040021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.040106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.040189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.040271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.040350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.040430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.040507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.040585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.040664] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.040780] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.040783] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54079, lrc_seqno=54079, guc_id=0, flags=0x73 in no process [-1]
<7> [178.040786] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.040844] ------------[ cut here ]------------
<4> [178.040846] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.040847] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.040919] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.040983] efi_pstore nfnetlink autofs4
<4> [178.040987] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.040990] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.040991] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.040992] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.040998] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.041069] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.041071] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.041073] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.041074] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.041075] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.041077] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.041078] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.041079] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.041081] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.041082] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.041084] PKRU: 55555554
<4> [178.041085] Call Trace:
<4> [178.041086] <TASK>
<4> [178.041090] ? lock_sync+0x100/0x100
<4> [178.041094] ? lock_release+0xd0/0x2b0
<4> [178.041099] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.041105] process_one_work+0x239/0x760
<4> [178.041111] worker_thread+0x200/0x3f0
<4> [178.041113] ? __pfx_worker_thread+0x10/0x10
<4> [178.041116] kthread+0x10d/0x150
<4> [178.041119] ? __pfx_kthread+0x10/0x10
<4> [178.041122] ret_from_fork+0x3d4/0x480
<4> [178.041124] ? __pfx_kthread+0x10/0x10
<4> [178.041128] ret_from_fork_asm+0x1a/0x30
<4> [178.041135] </TASK>
<4> [178.041136] irq event stamp: 527135
<4> [178.041137] hardirqs last enabled at (527141): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.041140] hardirqs last disabled at (527146): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.041142] softirqs last enabled at (526220): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.041144] softirqs last disabled at (526211): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.041147] ---[ end trace 0000000000000000 ]---
<6> [178.041148] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.041217] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.041223] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.042460] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.042566] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.042672] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.042886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.042978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.043073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.043161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.043248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.043335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.043421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.043509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.043591] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.043685] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.043809] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.044819] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.054760] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.055014] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.056275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.056360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.056439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.056518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.056596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.056672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.056747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.056844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.056921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.056999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.057075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.057150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.057225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.057300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.057375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.057450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.057524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.057597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.057673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.057762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.057845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.057925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.058005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.058084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.058161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.058238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.058315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.058396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.058476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.058555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.058633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.058713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.058810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.058895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.058980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.059061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.059144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.059224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.059304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.059382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.059462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.059540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.059616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.059696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.059789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.059869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.059948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.060026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.060104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.060186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.060266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.060348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.060427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.060506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.060583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.060662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.060737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.060825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.060906] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.061024] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.061028] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54079, lrc_seqno=54079, guc_id=0, flags=0x73 in no process [-1]
<7> [178.061031] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.061089] ------------[ cut here ]------------
<4> [178.061091] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.061092] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:9/209
<4> [178.061164] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.061230] efi_pstore nfnetlink autofs4
<4> [178.061235] CPU: 4 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.061238] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.061239] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.061240] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.061245] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.061315] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.061317] RSP: 0000:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.061319] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.061321] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.061322] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.061323] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.061324] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.061326] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [178.061327] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.061329] CR2: 0000768ff0098178 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [178.061330] PKRU: 55555554
<4> [178.061331] Call Trace:
<4> [178.061333] <TASK>
<4> [178.061336] ? lock_sync+0x100/0x100
<4> [178.061341] ? lock_release+0xd0/0x2b0
<4> [178.061347] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.061352] process_one_work+0x239/0x760
<4> [178.061358] worker_thread+0x200/0x3f0
<4> [178.061361] ? __pfx_worker_thread+0x10/0x10
<4> [178.061363] kthread+0x10d/0x150
<4> [178.061366] ? __pfx_kthread+0x10/0x10
<4> [178.061370] ret_from_fork+0x3d4/0x480
<4> [178.061372] ? __pfx_kthread+0x10/0x10
<4> [178.061375] ret_from_fork_asm+0x1a/0x30
<4> [178.061383] </TASK>
<4> [178.061384] irq event stamp: 2441685
<4> [178.061385] hardirqs last enabled at (2441691): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.061388] hardirqs last disabled at (2441696): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.061390] softirqs last enabled at (2440748): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.061393] softirqs last disabled at (2440741): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.061395] ---[ end trace 0000000000000000 ]---
<5> [178.062608] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54134, lrc_seqno=54134, guc_id=0, flags=0x73 in no process [-1]
<7> [178.062611] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.062671] ------------[ cut here ]------------
<4> [178.062672] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.062673] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:9/209
<4> [178.062743] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.062814] efi_pstore nfnetlink autofs4
<4> [178.062818] CPU: 4 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.062821] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.062822] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.062823] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.062828] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.062896] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.062898] RSP: 0000:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.062900] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.062901] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.062902] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.062904] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.062905] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.062906] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [178.062907] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.062909] CR2: 0000768ff0098178 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [178.062910] PKRU: 55555554
<4> [178.062911] Call Trace:
<4> [178.062912] <TASK>
<4> [178.062916] ? lock_sync+0x100/0x100
<4> [178.062920] ? lock_release+0xd0/0x2b0
<4> [178.062926] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.062931] process_one_work+0x239/0x760
<4> [178.062937] worker_thread+0x200/0x3f0
<4> [178.062939] ? __pfx_worker_thread+0x10/0x10
<4> [178.062942] kthread+0x10d/0x150
<4> [178.062945] ? __pfx_kthread+0x10/0x10
<4> [178.062948] ret_from_fork+0x3d4/0x480
<4> [178.062950] ? __pfx_kthread+0x10/0x10
<4> [178.062953] ret_from_fork_asm+0x1a/0x30
<4> [178.062961] </TASK>
<4> [178.062962] irq event stamp: 2443549
<4> [178.062963] hardirqs last enabled at (2443555): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.062965] hardirqs last disabled at (2443560): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.062968] softirqs last enabled at (2440748): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.062970] softirqs last disabled at (2440741): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.062972] ---[ end trace 0000000000000000 ]---
<7> [178.062987] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.063091] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.063191] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.063262] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.063268] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.063314] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.063517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.063604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.063698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.063800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.063888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.063974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.064060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.064148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.064231] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.064325] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.064441] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.065451] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.075759] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.076007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.077023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.077102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.077180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.077257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.077333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.077408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.077484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.077558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.077632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.077706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.077794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.077869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.077943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.078019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.078097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.078174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.078250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.078324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.078399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.078481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.078562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.078641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.078724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.078816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.078895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.078973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.079051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.079131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.079211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.079290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.079370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.079448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.079532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.079616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.079700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.079789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.079869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.079947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.080023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.080100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.080180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.080258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.080338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.080419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.080502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.080581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.080659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.080736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.080828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.080904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.080979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.081057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.081134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.081211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.081287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.081365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.081443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.081523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.081606] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.081719] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.081723] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54134, lrc_seqno=54134, guc_id=0, flags=0x73 in no process [-1]
<7> [178.081725] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.082646] ------------[ cut here ]------------
<4> [178.082647] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.082649] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:9/209
<4> [178.082727] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.082802] efi_pstore nfnetlink autofs4
<4> [178.082806] CPU: 4 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.082809] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.082810] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.082812] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.082817] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.082887] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.082889] RSP: 0000:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.082891] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.082893] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.082894] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.082895] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.082897] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.082898] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [178.082899] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.082901] CR2: 0000768ff0098178 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4> [178.082902] PKRU: 55555554
<4> [178.082903] Call Trace:
<4> [178.082905] <TASK>
<4> [178.082908] ? lock_sync+0x100/0x100
<4> [178.082913] ? lock_release+0xd0/0x2b0
<4> [178.082918] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.082924] process_one_work+0x239/0x760
<4> [178.082930] worker_thread+0x200/0x3f0
<4> [178.082933] ? __pfx_worker_thread+0x10/0x10
<4> [178.082935] kthread+0x10d/0x150
<4> [178.082938] ? __pfx_kthread+0x10/0x10
<4> [178.082942] ret_from_fork+0x3d4/0x480
<4> [178.082944] ? __pfx_kthread+0x10/0x10
<4> [178.082947] ret_from_fork_asm+0x1a/0x30
<4> [178.082954] </TASK>
<4> [178.082956] irq event stamp: 2446665
<4> [178.082957] hardirqs last enabled at (2446671): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.082960] hardirqs last disabled at (2446676): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.082962] softirqs last enabled at (2446088): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.082965] softirqs last disabled at (2446081): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.082967] ---[ end trace 0000000000000000 ]---
<7> [178.082980] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.083081] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.083178] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.083251] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.083259] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.083594] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.083803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.083892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.083984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.084073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.084160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.084245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.084332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.084421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.084509] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.084604] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.084720] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.085754] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.095757] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.096007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.097038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.097122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.097200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.097277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.097354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.097430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.097506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.097581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.097656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.097731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.097819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.097893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.097968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.098042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.098116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.098190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.098264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.098338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.098413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.098494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.098576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.098658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.098739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.098833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.098911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.098988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.099065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.099145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.099225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.099302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.099380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.099459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.099546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.099633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.099719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.099811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.099896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.099979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.100059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.100137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.100219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.100297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.100378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.100458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.100541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.100618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.100695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.100780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.100858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.100934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.101009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.101088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.101164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.101240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.101317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.101394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.101473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.101553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.101634] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.101754] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.101757] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54134, lrc_seqno=54134, guc_id=0, flags=0x73 in no process [-1]
<7> [178.101760] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.101819] ------------[ cut here ]------------
<4> [178.101820] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.101821] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.101894] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.101958] efi_pstore nfnetlink autofs4
<4> [178.101962] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.101965] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.101966] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.101968] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.101973] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.102043] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.102045] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.102047] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.102049] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.102050] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.102051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.102053] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.102054] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.102055] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.102057] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.102058] PKRU: 55555554
<4> [178.102060] Call Trace:
<4> [178.102061] <TASK>
<4> [178.102065] ? lock_sync+0x100/0x100
<4> [178.102069] ? lock_release+0xd0/0x2b0
<4> [178.102075] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.102081] process_one_work+0x239/0x760
<4> [178.102087] worker_thread+0x200/0x3f0
<4> [178.102090] ? __pfx_worker_thread+0x10/0x10
<4> [178.102093] kthread+0x10d/0x150
<4> [178.102096] ? __pfx_kthread+0x10/0x10
<4> [178.102100] ret_from_fork+0x3d4/0x480
<4> [178.102102] ? __pfx_kthread+0x10/0x10
<4> [178.102105] ret_from_fork_asm+0x1a/0x30
<4> [178.102113] </TASK>
<4> [178.102114] irq event stamp: 530571
<4> [178.102116] hardirqs last enabled at (530577): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.102118] hardirqs last disabled at (530582): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.102121] softirqs last enabled at (529474): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.102124] softirqs last disabled at (529469): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.102126] ---[ end trace 0000000000000000 ]---
<5> [178.103626] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54189, lrc_seqno=54189, guc_id=0, flags=0x73 in no process [-1]
<7> [178.103629] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.103692] ------------[ cut here ]------------
<4> [178.103693] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.103694] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.103779] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.103842] efi_pstore nfnetlink autofs4
<4> [178.103846] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.103849] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.103850] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.103852] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.103856] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.103925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.103927] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.103929] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.103930] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.103932] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.103933] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.103934] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.103935] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.103937] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.103938] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.103939] PKRU: 55555554
<4> [178.103941] Call Trace:
<4> [178.103942] <TASK>
<4> [178.103945] ? lock_sync+0x100/0x100
<4> [178.103949] ? lock_release+0xd0/0x2b0
<4> [178.103955] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.103960] process_one_work+0x239/0x760
<4> [178.103966] worker_thread+0x200/0x3f0
<4> [178.103969] ? __pfx_worker_thread+0x10/0x10
<4> [178.103971] kthread+0x10d/0x150
<4> [178.103974] ? __pfx_kthread+0x10/0x10
<4> [178.103977] ret_from_fork+0x3d4/0x480
<4> [178.103979] ? __pfx_kthread+0x10/0x10
<4> [178.103983] ret_from_fork_asm+0x1a/0x30
<4> [178.103990] </TASK>
<4> [178.103991] irq event stamp: 533001
<4> [178.103992] hardirqs last enabled at (533007): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.103995] hardirqs last disabled at (533012): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.103997] softirqs last enabled at (532258): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.103999] softirqs last disabled at (532251): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.104001] ---[ end trace 0000000000000000 ]---
<7> [178.104015] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.104119] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.104219] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.104293] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.104301] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.104556] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.104767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.104858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.104952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.105040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.105127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.105216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.105304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.105392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.105476] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.105570] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.105689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.106698] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.116755] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.117006] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.118011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.118091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.118168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.118247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.118325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.118404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.118481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.118556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.118630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.118705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.118794] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.118871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.118947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.119021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.119095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.119169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.119244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.119318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.119392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.119472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.119556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.119637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.119717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.119808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.119885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.119962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.120039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.120118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.120198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.120276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.120359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.120440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.120526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.120611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.120697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.120788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.120867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.120945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.121022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.121099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.121178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.121256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.121333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.121409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.121493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.121574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.121653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.121731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.121823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.121900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.121976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.122056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.122132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.122209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.122284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.122364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.122444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.122521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.122603] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.122720] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.122725] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54189, lrc_seqno=54189, guc_id=0, flags=0x73 in no process [-1]
<7> [178.122729] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.122841] ------------[ cut here ]------------
<4> [178.122843] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.122846] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.122983] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.123054] efi_pstore nfnetlink autofs4
<4> [178.123060] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.123063] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.123064] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.123067] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.123073] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.123152] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.123153] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.123156] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.123157] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.123158] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.123160] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.123161] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.123162] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.123164] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.123165] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.123167] PKRU: 55555554
<4> [178.123168] Call Trace:
<4> [178.123170] <TASK>
<4> [178.123176] ? lock_sync+0x100/0x100
<4> [178.123181] ? lock_release+0xd0/0x2b0
<4> [178.123187] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.123192] process_one_work+0x239/0x760
<4> [178.123199] worker_thread+0x200/0x3f0
<4> [178.123202] ? __pfx_worker_thread+0x10/0x10
<4> [178.123204] kthread+0x10d/0x150
<4> [178.123207] ? __pfx_kthread+0x10/0x10
<4> [178.123211] ret_from_fork+0x3d4/0x480
<4> [178.123214] ? __pfx_kthread+0x10/0x10
<4> [178.123217] ret_from_fork_asm+0x1a/0x30
<4> [178.123225] </TASK>
<4> [178.123226] irq event stamp: 136843
<4> [178.123227] hardirqs last enabled at (136849): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.123230] hardirqs last disabled at (136854): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.123232] softirqs last enabled at (136082): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.123235] softirqs last disabled at (136057): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.123237] ---[ end trace 0000000000000000 ]---
<6> [178.123240] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.123318] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.123327] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.123687] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.123812] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.123921] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.124124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.124213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.124307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.124396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.124485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.124574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.124662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.124758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.124841] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.124936] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.125052] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.126061] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.136750] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.137000] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.138026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.138107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.138186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.138263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.138342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.138420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.138497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.138572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.138647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.138721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.138806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.138884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.138961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.139037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.139113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.139188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.139262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.139337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.139411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.139496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.139578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.139659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.139739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.139830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.139909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.139990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.140070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.140152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.140234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.140312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.140391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.140470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.140558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.140644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.140730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.140825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.140906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.140984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.141063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.141140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.141219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.141296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.141373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.141449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.141530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.141609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.141690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.141775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.141856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.141933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.142009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.142089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.142167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.142244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.142321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.142403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.142483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.142562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.142641] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.142758] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.142762] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54189, lrc_seqno=54189, guc_id=0, flags=0x73 in no process [-1]
<7> [178.142765] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.142824] ------------[ cut here ]------------
<4> [178.142826] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.142827] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.142898] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.142962] efi_pstore nfnetlink autofs4
<4> [178.142966] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.142969] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.142971] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.142972] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.142977] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.143047] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.143049] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.143051] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.143052] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.143054] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.143055] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.143056] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.143057] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.143059] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.143060] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.143062] PKRU: 55555554
<4> [178.143063] Call Trace:
<4> [178.143064] <TASK>
<4> [178.143068] ? lock_sync+0x100/0x100
<4> [178.143072] ? lock_release+0xd0/0x2b0
<4> [178.143077] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.143083] process_one_work+0x239/0x760
<4> [178.143089] worker_thread+0x200/0x3f0
<4> [178.143092] ? __pfx_worker_thread+0x10/0x10
<4> [178.143094] kthread+0x10d/0x150
<4> [178.143097] ? __pfx_kthread+0x10/0x10
<4> [178.143101] ret_from_fork+0x3d4/0x480
<4> [178.143103] ? __pfx_kthread+0x10/0x10
<4> [178.143106] ret_from_fork_asm+0x1a/0x30
<4> [178.143113] </TASK>
<4> [178.143114] irq event stamp: 139965
<4> [178.143116] hardirqs last enabled at (139971): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.143118] hardirqs last disabled at (139976): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.143121] softirqs last enabled at (139170): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.143123] softirqs last disabled at (139147): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.143125] ---[ end trace 0000000000000000 ]---
<5> [178.143721] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54244, lrc_seqno=54244, guc_id=0, flags=0x73 in no process [-1]
<7> [178.143723] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.144903] ------------[ cut here ]------------
<4> [178.144904] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.144905] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.144981] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.145044] efi_pstore nfnetlink autofs4
<4> [178.145048] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.145051] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.145052] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.145054] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.145058] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.145130] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.145131] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.145133] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.145135] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.145136] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.145137] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.145138] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.145140] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.145141] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.145142] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.145144] PKRU: 55555554
<4> [178.145145] Call Trace:
<4> [178.145146] <TASK>
<4> [178.145150] ? lock_sync+0x100/0x100
<4> [178.145154] ? lock_release+0xd0/0x2b0
<4> [178.145159] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.145165] process_one_work+0x239/0x760
<4> [178.145171] worker_thread+0x200/0x3f0
<4> [178.145174] ? __pfx_worker_thread+0x10/0x10
<4> [178.145176] kthread+0x10d/0x150
<4> [178.145179] ? __pfx_kthread+0x10/0x10
<4> [178.145182] ret_from_fork+0x3d4/0x480
<4> [178.145184] ? __pfx_kthread+0x10/0x10
<4> [178.145188] ret_from_fork_asm+0x1a/0x30
<4> [178.145195] </TASK>
<4> [178.145196] irq event stamp: 142683
<4> [178.145197] hardirqs last enabled at (142689): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.145200] hardirqs last disabled at (142694): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.145202] softirqs last enabled at (141918): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.145204] softirqs last disabled at (141907): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.145206] ---[ end trace 0000000000000000 ]---
<6> [178.145208] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.145279] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.145284] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.145299] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.145405] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.145514] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.145721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.145852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.145961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.146051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.146140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.146226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.146314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.146404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.146489] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.146585] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.146704] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.147725] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.157751] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.157999] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.159021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.159103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.159181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.159258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.159335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.159411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.159487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.159562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.159637] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.159712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.159800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.159890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.159965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.160040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.160115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.160189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.160264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.160342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.160419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.160503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.160585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.160664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.160751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.160843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.160923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.161004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.161083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.161163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.161244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.161323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.161403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.161487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.161576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.161667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.161762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.161868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.161950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.162030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.162113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.162192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.162273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.162353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.162432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.162509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.162591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.162669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.162755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.162846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.162925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.163001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.163078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.163162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.163243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.163322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.163400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.163478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.163555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.163632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.163712] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.163849] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.163856] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54244, lrc_seqno=54244, guc_id=0, flags=0x73 in no process [-1]
<7> [178.163860] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.163925] ------------[ cut here ]------------
<4> [178.163926] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.163928] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.164000] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.164065] efi_pstore nfnetlink autofs4
<4> [178.164069] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.164072] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.164073] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.164075] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.164080] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.164150] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.164151] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.164154] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.164155] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.164156] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.164158] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.164159] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.164160] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.164162] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.164163] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.164165] PKRU: 55555554
<4> [178.164166] Call Trace:
<4> [178.164167] <TASK>
<4> [178.164171] ? lock_sync+0x100/0x100
<4> [178.164175] ? lock_release+0xd0/0x2b0
<4> [178.164181] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.164187] process_one_work+0x239/0x760
<4> [178.164193] worker_thread+0x200/0x3f0
<4> [178.164196] ? __pfx_worker_thread+0x10/0x10
<4> [178.164198] kthread+0x10d/0x150
<4> [178.164201] ? __pfx_kthread+0x10/0x10
<4> [178.164205] ret_from_fork+0x3d4/0x480
<4> [178.164207] ? __pfx_kthread+0x10/0x10
<4> [178.164210] ret_from_fork_asm+0x1a/0x30
<4> [178.164217] </TASK>
<4> [178.164219] irq event stamp: 538387
<4> [178.164220] hardirqs last enabled at (538393): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.164222] hardirqs last disabled at (538398): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.164225] softirqs last enabled at (537552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.164227] softirqs last disabled at (537535): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.164229] ---[ end trace 0000000000000000 ]---
<6> [178.164231] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.164300] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.164306] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.164714] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.164935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.165024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.165118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.165207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.165298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.165386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.165473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.165561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.165644] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.165747] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.165865] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.166874] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.177596] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.177848] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.178866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.178947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.179024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.179104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.179181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.179256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.179334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.179412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.179488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.179564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.179640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.179714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.179802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.179875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.179949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.180023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.180098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.180172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.180245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.180327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.180409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.180491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.180572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.180650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.180728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.180818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.180895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.180974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.181055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.181133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.181211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.181290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.181374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.181458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.181541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.181626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.181711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.181799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.181879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.181957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.182038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.182116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.182194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.182271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.182355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.182437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.182517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.182595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.182674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.182753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.182830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.182910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.182987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.183065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.183147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.183228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.183306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.183387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.183470] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.183581] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.183585] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54244, lrc_seqno=54244, guc_id=0, flags=0x73 in no process [-1]
<7> [178.183587] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.183645] ------------[ cut here ]------------
<4> [178.183646] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.183648] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.183719] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.183788] efi_pstore nfnetlink autofs4
<4> [178.183792] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.183795] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.183796] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.183797] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.183802] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.183871] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.183873] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.183875] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.183877] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.183878] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.183879] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.183880] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.183882] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.183883] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.183885] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.183886] PKRU: 55555554
<4> [178.183887] Call Trace:
<4> [178.183888] <TASK>
<4> [178.183892] ? lock_sync+0x100/0x100
<4> [178.183897] ? lock_release+0xd0/0x2b0
<4> [178.183902] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.183908] process_one_work+0x239/0x760
<4> [178.183914] worker_thread+0x200/0x3f0
<4> [178.183916] ? __pfx_worker_thread+0x10/0x10
<4> [178.183919] kthread+0x10d/0x150
<4> [178.183922] ? __pfx_kthread+0x10/0x10
<4> [178.183925] ret_from_fork+0x3d4/0x480
<4> [178.183927] ? __pfx_kthread+0x10/0x10
<4> [178.183931] ret_from_fork_asm+0x1a/0x30
<4> [178.183938] </TASK>
<4> [178.183939] irq event stamp: 541473
<4> [178.183940] hardirqs last enabled at (541479): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.183943] hardirqs last disabled at (541484): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.183945] softirqs last enabled at (540870): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.183948] softirqs last disabled at (540865): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.183950] ---[ end trace 0000000000000000 ]---
<7> [178.185489] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.185594] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [178.185692] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54299, lrc_seqno=54299, guc_id=0, flags=0x73 in no process [-1]
<7> [178.185695] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.185767] ------------[ cut here ]------------
<4> [178.185768] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.185770] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.185843] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.185906] efi_pstore nfnetlink autofs4
<4> [178.185910] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.185913] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.185914] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.185915] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.185920] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.185992] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.185994] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.185996] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.185997] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.185998] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.186000] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.186001] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.186002] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.186004] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.186005] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.186006] PKRU: 55555554
<4> [178.186007] Call Trace:
<4> [178.186008] <TASK>
<4> [178.186012] ? lock_sync+0x100/0x100
<4> [178.186016] ? lock_release+0xd0/0x2b0
<4> [178.186022] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.186028] process_one_work+0x239/0x760
<4> [178.186033] worker_thread+0x200/0x3f0
<4> [178.186036] ? __pfx_worker_thread+0x10/0x10
<4> [178.186039] kthread+0x10d/0x150
<4> [178.186042] ? __pfx_kthread+0x10/0x10
<4> [178.186045] ret_from_fork+0x3d4/0x480
<4> [178.186047] ? __pfx_kthread+0x10/0x10
<4> [178.186050] ret_from_fork_asm+0x1a/0x30
<4> [178.186058] </TASK>
<4> [178.186059] irq event stamp: 544229
<4> [178.186060] hardirqs last enabled at (544235): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.186062] hardirqs last disabled at (544240): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.186065] softirqs last enabled at (540870): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.186067] softirqs last disabled at (540865): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.186069] ---[ end trace 0000000000000000 ]---
<6> [178.186071] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.186142] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.186147] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.186194] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.186395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.186485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.186578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.186667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.186756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.186840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.186926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.187017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.187101] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.187199] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.187316] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.188338] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.198742] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.198993] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.199984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.200066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.200144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.200223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.200300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.200377] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.200453] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.200529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.200605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.200680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.200760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.200836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.200911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.200986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.201061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.201137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.201215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.201292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.201367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.201449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.201530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.201609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.201689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.201776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.201853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.201930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.202007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.202086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.202168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.202251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.202333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.202414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.202500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.202585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.202676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.202764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.202846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.202925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.203004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.203080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.203161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.203238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.203315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.203391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.203471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.203548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.203626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.203702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.203788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.203864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.203941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.204019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.204096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.204172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.204251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.204330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.204407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.204483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.204563] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.204673] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.204677] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54299, lrc_seqno=54299, guc_id=0, flags=0x73 in no process [-1]
<7> [178.204679] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.205586] ------------[ cut here ]------------
<4> [178.205588] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.205589] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.205663] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.205726] efi_pstore nfnetlink autofs4
<4> [178.205730] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.205733] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.205738] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.205739] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.205744] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.205814] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.205816] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.205818] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.205819] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.205821] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.205822] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.205823] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.205824] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.205826] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.205827] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.205829] PKRU: 55555554
<4> [178.205830] Call Trace:
<4> [178.205831] <TASK>
<4> [178.205835] ? lock_sync+0x100/0x100
<4> [178.205839] ? lock_release+0xd0/0x2b0
<4> [178.205844] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.205850] process_one_work+0x239/0x760
<4> [178.205856] worker_thread+0x200/0x3f0
<4> [178.205859] ? __pfx_worker_thread+0x10/0x10
<4> [178.205861] kthread+0x10d/0x150
<4> [178.205864] ? __pfx_kthread+0x10/0x10
<4> [178.205868] ret_from_fork+0x3d4/0x480
<4> [178.205870] ? __pfx_kthread+0x10/0x10
<4> [178.205873] ret_from_fork_asm+0x1a/0x30
<4> [178.205880] </TASK>
<4> [178.205882] irq event stamp: 547333
<4> [178.205883] hardirqs last enabled at (547339): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.205886] hardirqs last disabled at (547344): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.205888] softirqs last enabled at (547150): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.205890] softirqs last disabled at (547145): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.205893] ---[ end trace 0000000000000000 ]---
<7> [178.205906] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.206007] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.206107] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.206179] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.206185] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.206590] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.206802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.206895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.206990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.207079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.207168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.207258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.207346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.207434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.207517] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.207612] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.207731] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.208991] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.218748] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.218997] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.220011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.220091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.220169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.220247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.220327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.220405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.220481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.220557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.220632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.220706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.220789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.220864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.220938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.221012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.221086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.221159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.221232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.221306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.221379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.221461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.221543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.221626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.221708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.221797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.221876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.221952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.222029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.222109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.222189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.222268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.222347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.222426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.222510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.222594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.222677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.222764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.222845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.222928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.223008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.223086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.223167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.223246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.223324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.223401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.223481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.223556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.223633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.223710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.223796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.223873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.223954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.224035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.224114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.224192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.224269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.224348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.224424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.224500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.224578] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.224688] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.224692] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54299, lrc_seqno=54299, guc_id=0, flags=0x73 in no process [-1]
<7> [178.224695] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.224764] ------------[ cut here ]------------
<4> [178.224765] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.224767] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.224837] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.224902] efi_pstore nfnetlink autofs4
<4> [178.224906] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.224909] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.224910] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.224912] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.224917] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.224987] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.224988] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.224991] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.224992] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.224993] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.224994] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.224996] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.224997] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.224999] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.225000] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.225002] PKRU: 55555554
<4> [178.225003] Call Trace:
<4> [178.225004] <TASK>
<4> [178.225008] ? lock_sync+0x100/0x100
<4> [178.225012] ? lock_release+0xd0/0x2b0
<4> [178.225018] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.225023] process_one_work+0x239/0x760
<4> [178.225030] worker_thread+0x200/0x3f0
<4> [178.225033] ? __pfx_worker_thread+0x10/0x10
<4> [178.225035] kthread+0x10d/0x150
<4> [178.225038] ? __pfx_kthread+0x10/0x10
<4> [178.225042] ret_from_fork+0x3d4/0x480
<4> [178.225044] ? __pfx_kthread+0x10/0x10
<4> [178.225047] ret_from_fork_asm+0x1a/0x30
<4> [178.225054] </TASK>
<4> [178.225055] irq event stamp: 146493
<4> [178.225057] hardirqs last enabled at (146499): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.225059] hardirqs last disabled at (146504): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.225062] softirqs last enabled at (145584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.225064] softirqs last disabled at (145577): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.225066] ---[ end trace 0000000000000000 ]---
<5> [178.226556] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54354, lrc_seqno=54354, guc_id=0, flags=0x73 in no process [-1]
<7> [178.226560] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.226627] ------------[ cut here ]------------
<4> [178.226628] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.226629] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.226699] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.226767] efi_pstore nfnetlink autofs4
<4> [178.226771] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.226774] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.226775] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.226776] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.226781] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.226852] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.226854] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.226856] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.226857] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.226858] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.226860] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.226861] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.226862] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.226864] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.226865] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.226866] PKRU: 55555554
<4> [178.226868] Call Trace:
<4> [178.226869] <TASK>
<4> [178.226872] ? lock_sync+0x100/0x100
<4> [178.226877] ? lock_release+0xd0/0x2b0
<4> [178.226882] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.226888] process_one_work+0x239/0x760
<4> [178.226894] worker_thread+0x200/0x3f0
<4> [178.226896] ? __pfx_worker_thread+0x10/0x10
<4> [178.226899] kthread+0x10d/0x150
<4> [178.226902] ? __pfx_kthread+0x10/0x10
<4> [178.226905] ret_from_fork+0x3d4/0x480
<4> [178.226907] ? __pfx_kthread+0x10/0x10
<4> [178.226910] ret_from_fork_asm+0x1a/0x30
<4> [178.226918] </TASK>
<4> [178.226919] irq event stamp: 149217
<4> [178.226920] hardirqs last enabled at (149223): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.226923] hardirqs last disabled at (149228): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.226925] softirqs last enabled at (145584): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.226927] softirqs last disabled at (145577): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.226930] ---[ end trace 0000000000000000 ]---
<7> [178.226943] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.227047] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.227146] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.227218] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.227224] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.227270] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.227475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.227564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.227657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.227755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.227847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.227936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.228025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.228114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.228197] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.228290] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.228407] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.229415] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.239738] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.239988] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.241007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.241087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.241164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.241241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.241317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.241393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.241469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.241546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.241622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.241701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.241789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.241865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.241941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.242016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.242092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.242167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.242241] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.242316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.242394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.242478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.242562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.242641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.242721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.242815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.242893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.242971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.243049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.243130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.243211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.243291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.243371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.243450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.243535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.243620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.243708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.243802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.243884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.243963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.244043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.244119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.244198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.244277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.244353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.244430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.244510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.244586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.244662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.244742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.244824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.244901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.244978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.245059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.245136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.245213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.245290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.245368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.245445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.245525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.245608] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.245718] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.245722] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54354, lrc_seqno=54354, guc_id=0, flags=0x73 in no process [-1]
<7> [178.245724] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.246649] ------------[ cut here ]------------
<4> [178.246650] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.246651] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.246726] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.246794] efi_pstore nfnetlink autofs4
<4> [178.246798] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.246801] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.246802] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.246803] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.246808] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.246878] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.246880] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.246882] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.246883] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.246885] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.246886] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.246887] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.246888] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.246890] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.246891] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.246893] PKRU: 55555554
<4> [178.246894] Call Trace:
<4> [178.246895] <TASK>
<4> [178.246899] ? lock_sync+0x100/0x100
<4> [178.246903] ? lock_release+0xd0/0x2b0
<4> [178.246909] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.246914] process_one_work+0x239/0x760
<4> [178.246920] worker_thread+0x200/0x3f0
<4> [178.246923] ? __pfx_worker_thread+0x10/0x10
<4> [178.246925] kthread+0x10d/0x150
<4> [178.246928] ? __pfx_kthread+0x10/0x10
<4> [178.246932] ret_from_fork+0x3d4/0x480
<4> [178.246934] ? __pfx_kthread+0x10/0x10
<4> [178.246937] ret_from_fork_asm+0x1a/0x30
<4> [178.246945] </TASK>
<4> [178.246946] irq event stamp: 152357
<4> [178.246947] hardirqs last enabled at (152363): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.246950] hardirqs last disabled at (152368): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.246952] softirqs last enabled at (151436): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.246954] softirqs last disabled at (151429): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.246957] ---[ end trace 0000000000000000 ]---
<6> [178.246958] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.247028] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.247034] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.247048] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.247150] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.247439] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.247645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.247742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.247838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.247929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.248018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.248105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.248193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.248281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.248365] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.248461] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.248578] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.249601] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.259735] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.259985] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.260986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.261066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.261143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.261221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.261297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.261373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.261449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.261524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.261599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.261674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.261754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.261829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.261905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.261980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.262055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.262132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.262211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.262287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.262363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.262446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.262527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.262607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.262688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.262775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.262852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.262932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.263013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.263096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.263177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.263257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.263336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.263415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.263499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.263583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.263667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.263750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.263831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.263912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.263991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.264069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.264150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.264229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.264306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.264387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.264471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.264551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.264629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.264706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.264793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.264869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.264945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.265023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.265100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.265177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.265253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.265331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.265407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.265482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.265561] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.265671] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.265675] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54354, lrc_seqno=54354, guc_id=0, flags=0x73 in no process [-1]
<7> [178.265678] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.266589] ------------[ cut here ]------------
<4> [178.266590] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.266592] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.266666] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.266735] efi_pstore nfnetlink autofs4
<4> [178.266739] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.266742] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.266743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.266745] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.266750] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.266822] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.266823] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.266825] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.266827] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.266828] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.266829] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.266831] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.266832] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.266834] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.266835] CR2: 0000758009e44e78 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.266837] PKRU: 55555554
<4> [178.266838] Call Trace:
<4> [178.266839] <TASK>
<4> [178.266843] ? lock_sync+0x100/0x100
<4> [178.266847] ? lock_release+0xd0/0x2b0
<4> [178.266853] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.266858] process_one_work+0x239/0x760
<4> [178.266865] worker_thread+0x200/0x3f0
<4> [178.266868] ? __pfx_worker_thread+0x10/0x10
<4> [178.266870] kthread+0x10d/0x150
<4> [178.266873] ? __pfx_kthread+0x10/0x10
<4> [178.266877] ret_from_fork+0x3d4/0x480
<4> [178.266879] ? __pfx_kthread+0x10/0x10
<4> [178.266882] ret_from_fork_asm+0x1a/0x30
<4> [178.266890] </TASK>
<4> [178.266891] irq event stamp: 550739
<4> [178.266892] hardirqs last enabled at (550745): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.266895] hardirqs last disabled at (550750): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.266897] softirqs last enabled at (549972): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.266899] softirqs last disabled at (549959): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.266901] ---[ end trace 0000000000000000 ]---
<5> [178.267495] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54409, lrc_seqno=54409, guc_id=0, flags=0x73 in no process [-1]
<7> [178.267498] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.267557] ------------[ cut here ]------------
<4> [178.267558] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.267559] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.267628] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.267691] efi_pstore nfnetlink autofs4
<4> [178.267696] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.267698] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.267699] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.267700] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.267705] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.267783] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.267784] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.267787] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.267788] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.267789] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.267791] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.267792] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.267793] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.267795] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.267796] CR2: 0000758009e44e78 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.267798] PKRU: 55555554
<4> [178.267799] Call Trace:
<4> [178.267800] <TASK>
<4> [178.267803] ? lock_sync+0x100/0x100
<4> [178.267808] ? lock_release+0xd0/0x2b0
<4> [178.267813] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.267819] process_one_work+0x239/0x760
<4> [178.267824] worker_thread+0x200/0x3f0
<4> [178.267827] ? __pfx_worker_thread+0x10/0x10
<4> [178.267829] kthread+0x10d/0x150
<4> [178.267832] ? __pfx_kthread+0x10/0x10
<4> [178.267836] ret_from_fork+0x3d4/0x480
<4> [178.267838] ? __pfx_kthread+0x10/0x10
<4> [178.267841] ret_from_fork_asm+0x1a/0x30
<4> [178.267849] </TASK>
<4> [178.267850] irq event stamp: 553457
<4> [178.267851] hardirqs last enabled at (553463): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.267853] hardirqs last disabled at (553468): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.267856] softirqs last enabled at (549972): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.267858] softirqs last disabled at (549959): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.267860] ---[ end trace 0000000000000000 ]---
<6> [178.267862] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.267931] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.267937] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.267951] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.268055] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.268169] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.268379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.268473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.268567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.268656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.268747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.268833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.268919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.269007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.269089] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.269185] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.269302] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.270310] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.280733] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.280982] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.282006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.282087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.282165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.282242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.282318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.282395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.282474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.282550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.282627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.282705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.282792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.282870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.282947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.283023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.283098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.283172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.283247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.283325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.283402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.283485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.283568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.283648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.283737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.283818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.283897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.283975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.284051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.284132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.284212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.284291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.284370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.284448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.284535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.284623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.284711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.284807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.284889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.284969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.285052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.285132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.285214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.285294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.285373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.285450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.285532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.285609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.285687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.285772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.285850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.285926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.286002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.286081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.286158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.286233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.286311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.286394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.286473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.286551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.286630] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.286746] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.286751] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54409, lrc_seqno=54409, guc_id=0, flags=0x73 in no process [-1]
<7> [178.286753] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.286813] ------------[ cut here ]------------
<4> [178.286814] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.286816] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.286888] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.286953] efi_pstore nfnetlink autofs4
<4> [178.286957] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.286960] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.286961] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.286963] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.286968] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.287038] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.287039] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.287042] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.287043] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.287045] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.287046] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.287047] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.287049] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.287050] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.287052] CR2: 000060dda491f42c CR3: 000000012cae9001 CR4: 0000000000f72ef0
<4> [178.287053] PKRU: 55555554
<4> [178.287054] Call Trace:
<4> [178.287055] <TASK>
<4> [178.287060] ? lock_sync+0x100/0x100
<4> [178.287064] ? lock_release+0xd0/0x2b0
<4> [178.287070] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.287076] process_one_work+0x239/0x760
<4> [178.287082] worker_thread+0x200/0x3f0
<4> [178.287085] ? __pfx_worker_thread+0x10/0x10
<4> [178.287088] kthread+0x10d/0x150
<4> [178.287090] ? __pfx_kthread+0x10/0x10
<4> [178.287094] ret_from_fork+0x3d4/0x480
<4> [178.287096] ? __pfx_kthread+0x10/0x10
<4> [178.287100] ret_from_fork_asm+0x1a/0x30
<4> [178.287107] </TASK>
<4> [178.287108] irq event stamp: 2461295
<4> [178.287109] hardirqs last enabled at (2461301): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.287112] hardirqs last disabled at (2461306): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.287115] softirqs last enabled at (2460380): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.287117] softirqs last disabled at (2460373): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.287119] ---[ end trace 0000000000000000 ]---
<6> [178.287121] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.287190] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.287196] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.288432] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.288536] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.288641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.288852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.288944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.289039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.289130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.289220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.289309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.289397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.289485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.289567] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.289661] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.289785] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.290795] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.300731] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.300980] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.301999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.302080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.302157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.302234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.302310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.302386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.302461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.302537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.302612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.302686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.302775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.302851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.302926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.303002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.303077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.303152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.303226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.303300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.303375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.303456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.303538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.303618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.303698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.303788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.303871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.303952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.304031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.304112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.304192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.304270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.304349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.304428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.304511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.304595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.304680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.304769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.304850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.304933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.305014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.305094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.305174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.305252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.305330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.305407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.305488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.305569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.305649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.305730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.305810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.305885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.305962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.306042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.306123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.306204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.306283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.306362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.306439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.306516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.306596] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.306707] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.306711] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54409, lrc_seqno=54409, guc_id=0, flags=0x73 in no process [-1]
<7> [178.306713] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.306794] ------------[ cut here ]------------
<4> [178.306796] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.306797] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:39/3057
<4> [178.306868] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.306932] efi_pstore nfnetlink autofs4
<4> [178.306937] CPU: 6 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.306940] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.306941] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.306942] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.306948] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.307017] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.307018] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.307020] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.307022] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.307023] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.307024] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.307026] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.307027] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.307029] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.307030] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.307032] PKRU: 55555554
<4> [178.307033] Call Trace:
<4> [178.307034] <TASK>
<4> [178.307038] ? lock_sync+0x100/0x100
<4> [178.307042] ? lock_release+0xd0/0x2b0
<4> [178.307048] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.307053] process_one_work+0x239/0x760
<4> [178.307060] worker_thread+0x200/0x3f0
<4> [178.307062] ? __pfx_worker_thread+0x10/0x10
<4> [178.307065] kthread+0x10d/0x150
<4> [178.307067] ? __pfx_kthread+0x10/0x10
<4> [178.307071] ret_from_fork+0x3d4/0x480
<4> [178.307073] ? __pfx_kthread+0x10/0x10
<4> [178.307076] ret_from_fork_asm+0x1a/0x30
<4> [178.307084] </TASK>
<4> [178.307085] irq event stamp: 155907
<4> [178.307086] hardirqs last enabled at (155913): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.307089] hardirqs last disabled at (155918): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.307091] softirqs last enabled at (155146): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.307093] softirqs last disabled at (155123): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.307095] ---[ end trace 0000000000000000 ]---
<5> [178.308731] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54464, lrc_seqno=54464, guc_id=0, flags=0x73 in no process [-1]
<7> [178.308734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.308799] ------------[ cut here ]------------
<4> [178.308801] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.308802] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.308874] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.308939] efi_pstore nfnetlink autofs4
<4> [178.308944] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.308946] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.308947] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.308949] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [178.308926] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [178.308954] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.309024] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.309025] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.309027] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.309029] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.309030] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.309031] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.309032] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.309033] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.309035] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.309037] CR2: 000060dda491f42c CR3: 000000012cae9001 CR4: 0000000000f72ef0
<4> [178.309038] PKRU: 55555554
<4> [178.309039] Call Trace:
<4> [178.309040] <TASK>
<4> [178.309044] ? lock_sync+0x100/0x100
<4> [178.309048] ? lock_release+0xd0/0x2b0
<4> [178.309054] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.309059] process_one_work+0x239/0x760
<4> [178.309065] worker_thread+0x200/0x3f0
<4> [178.309068] ? __pfx_worker_thread+0x10/0x10
<4> [178.309070] kthread+0x10d/0x150
<4> [178.309073] ? __pfx_kthread+0x10/0x10
<4> [178.309077] ret_from_fork+0x3d4/0x480
<4> [178.309079] ? __pfx_kthread+0x10/0x10
<4> [178.309082] ret_from_fork_asm+0x1a/0x30
<7> [178.309037] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [178.309089] </TASK>
<4> [178.309091] irq event stamp: 158701
<4> [178.309091] hardirqs last enabled at (158707): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.309094] hardirqs last disabled at (158712): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.309096] softirqs last enabled at (155146): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.309098] softirqs last disabled at (155123): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.309100] ---[ end trace 0000000000000000 ]---
<6> [178.309102] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.309173] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.309179] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.309271] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.309473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.309563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.309656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.309750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.309841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.309930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.310019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.310109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.310192] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.310288] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.310407] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.311415] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.322077] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.322325] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.323611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.323754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.323876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.323998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.324118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.324238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.324359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.324479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.324599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.324719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.324847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.324978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.325058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.325136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.325213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.325289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.325365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.325441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.325516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.325599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.325681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.325770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.325855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.325934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.326013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.326090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.326168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.326248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.326328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.326406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.326487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.326570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.326657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.326745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.326831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.326913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.326992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.327071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.327148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.327226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.327306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.327387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.327469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.327547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.327629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.327707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.327796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.327873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.327950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.328026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.328103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.328187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.328269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.328350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.328428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.328507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.328585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.328661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.328746] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.328859] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.328864] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54464, lrc_seqno=54464, guc_id=0, flags=0x73 in no process [-1]
<7> [178.328866] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.328925] ------------[ cut here ]------------
<4> [178.328926] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.328928] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.329000] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.329063] efi_pstore nfnetlink autofs4
<4> [178.329067] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.329070] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.329071] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.329073] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.329077] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.329147] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.329148] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.329151] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.329152] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.329153] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.329154] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.329155] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.329157] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.329158] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.329160] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.329161] PKRU: 55555554
<4> [178.329163] Call Trace:
<4> [178.329164] <TASK>
<4> [178.329168] ? lock_sync+0x100/0x100
<4> [178.329172] ? lock_release+0xd0/0x2b0
<4> [178.329178] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.329183] process_one_work+0x239/0x760
<4> [178.329189] worker_thread+0x200/0x3f0
<4> [178.329192] ? __pfx_worker_thread+0x10/0x10
<4> [178.329194] kthread+0x10d/0x150
<4> [178.329197] ? __pfx_kthread+0x10/0x10
<4> [178.329201] ret_from_fork+0x3d4/0x480
<4> [178.329203] ? __pfx_kthread+0x10/0x10
<4> [178.329206] ret_from_fork_asm+0x1a/0x30
<4> [178.329213] </TASK>
<4> [178.329215] irq event stamp: 161797
<4> [178.329216] hardirqs last enabled at (161803): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.329218] hardirqs last disabled at (161808): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.329221] softirqs last enabled at (160954): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.329223] softirqs last disabled at (160945): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.329225] ---[ end trace 0000000000000000 ]---
<6> [178.329227] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.329297] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.329303] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.329473] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.329673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.329771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.329867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.329958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.330049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.330137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.330225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.330313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.330399] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.330495] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.330615] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.331624] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.341726] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.341973] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.342979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.343058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.343135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.343213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.343292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.343373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.343451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.343526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.343601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.343676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.343760] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.343834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.343908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.343982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.344055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.344129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.344202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.344276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.344353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.344436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.344519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.344599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.344679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.344764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.344841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.344919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.344996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.345077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.345158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.345237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.345317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.345401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.345488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.345575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.345660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.345744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.345826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.345904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.345983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.346059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.346140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.346218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.346296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.346372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.346453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.346529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.346608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.346689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.346786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.346866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.346944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.347025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.347103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.347181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.347257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.347336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.347412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.347488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.347566] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.347675] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.347679] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54464, lrc_seqno=54464, guc_id=0, flags=0x73 in no process [-1]
<7> [178.347681] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.348595] ------------[ cut here ]------------
<4> [178.348597] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.348598] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.348670] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.348737] efi_pstore nfnetlink autofs4
<4> [178.348742] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.348744] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.348746] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.348747] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.348752] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.348821] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.348823] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.348825] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.348827] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.348828] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.348829] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.348830] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.348832] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.348833] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.348834] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.348836] PKRU: 55555554
<4> [178.348837] Call Trace:
<4> [178.348838] <TASK>
<4> [178.348842] ? lock_sync+0x100/0x100
<4> [178.348846] ? lock_release+0xd0/0x2b0
<4> [178.348852] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.348857] process_one_work+0x239/0x760
<4> [178.348863] worker_thread+0x200/0x3f0
<4> [178.348866] ? __pfx_worker_thread+0x10/0x10
<4> [178.348868] kthread+0x10d/0x150
<4> [178.348871] ? __pfx_kthread+0x10/0x10
<4> [178.348875] ret_from_fork+0x3d4/0x480
<4> [178.348877] ? __pfx_kthread+0x10/0x10
<4> [178.348880] ret_from_fork_asm+0x1a/0x30
<4> [178.348887] </TASK>
<4> [178.348888] irq event stamp: 164947
<4> [178.348889] hardirqs last enabled at (164953): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.348892] hardirqs last disabled at (164958): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.348894] softirqs last enabled at (164180): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.348897] softirqs last disabled at (164173): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.348899] ---[ end trace 0000000000000000 ]---
<7> [178.348912] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.349016] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [178.349723] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54519, lrc_seqno=54519, guc_id=0, flags=0x73 in no process [-1]
<7> [178.349730] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.349838] ------------[ cut here ]------------
<4> [178.349840] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.349842] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:9/209
<4> [178.349960] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.350051] efi_pstore nfnetlink autofs4
<4> [178.350057] CPU: 12 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.350061] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.350063] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.350065] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.350072] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.350186] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.350188] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.350191] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.350193] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.350195] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.350197] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.350199] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.350201] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [178.350203] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.350205] CR2: 0000768ff00973c8 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [178.350208] PKRU: 55555554
<4> [178.350209] Call Trace:
<4> [178.350211] <TASK>
<4> [178.350217] ? lock_sync+0x100/0x100
<4> [178.350224] ? lock_release+0xd0/0x2b0
<4> [178.350233] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.350243] process_one_work+0x239/0x760
<4> [178.350253] worker_thread+0x200/0x3f0
<4> [178.350257] ? __pfx_worker_thread+0x10/0x10
<4> [178.350260] kthread+0x10d/0x150
<4> [178.350264] ? __pfx_kthread+0x10/0x10
<4> [178.350269] ret_from_fork+0x3d4/0x480
<4> [178.350272] ? __pfx_kthread+0x10/0x10
<4> [178.350277] ret_from_fork_asm+0x1a/0x30
<4> [178.350289] </TASK>
<4> [178.350291] irq event stamp: 2464173
<4> [178.350292] hardirqs last enabled at (2464179): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.350296] hardirqs last disabled at (2464184): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.350299] softirqs last enabled at (2463400): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.350303] softirqs last disabled at (2463393): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.350306] ---[ end trace 0000000000000000 ]---
<6> [178.350308] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.350421] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.350430] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.350723] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.350974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.351108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.351246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.351379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.351511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.351642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.351780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.351916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.352045] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.352187] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.352352] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.353756] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.363729] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.364037] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.365198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.365318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.365437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.365558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.365678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.365806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.365927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.366047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.366167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.366287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.366407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.366528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.366648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.366773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.366894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.367014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.367134] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.367254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.367375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.367499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.367624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.367753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.367880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.368004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.368128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.368251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.368376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.368503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.368630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.368759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.368886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.369011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.369143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.369274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.369405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.369533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.369661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.369838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.369930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.370014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.370097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.370176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.370255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.370332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.370413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.370489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.370567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.370644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.370730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.370813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.370893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.370975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.371053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.371132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.371209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.371288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.371364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.371440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.371519] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.371635] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.371639] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54519, lrc_seqno=54519, guc_id=0, flags=0x73 in no process [-1]
<7> [178.371642] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.371701] ------------[ cut here ]------------
<4> [178.371703] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.371704] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:9/209
<4> [178.371791] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.371856] efi_pstore nfnetlink autofs4
<4> [178.371861] CPU: 4 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.371863] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.371865] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.371866] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.371871] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.371941] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.371943] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.371945] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.371946] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.371948] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.371949] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.371950] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.371951] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000
<4> [178.371953] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.371954] CR2: 0000768ff0098178 CR3: 000000012cae9002 CR4: 0000000000f72ef0
<4> [178.371956] PKRU: 55555554
<4> [178.371957] Call Trace:
<4> [178.371958] <TASK>
<4> [178.371962] ? lock_sync+0x100/0x100
<4> [178.371966] ? lock_release+0xd0/0x2b0
<4> [178.371972] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.371977] process_one_work+0x239/0x760
<4> [178.371984] worker_thread+0x200/0x3f0
<4> [178.371986] ? __pfx_worker_thread+0x10/0x10
<4> [178.371989] kthread+0x10d/0x150
<4> [178.371992] ? __pfx_kthread+0x10/0x10
<4> [178.371995] ret_from_fork+0x3d4/0x480
<4> [178.371997] ? __pfx_kthread+0x10/0x10
<4> [178.372001] ret_from_fork_asm+0x1a/0x30
<4> [178.372008] </TASK>
<4> [178.372009] irq event stamp: 2467327
<4> [178.372010] hardirqs last enabled at (2467333): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.372013] hardirqs last disabled at (2467338): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.372015] softirqs last enabled at (2466418): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.372017] softirqs last disabled at (2466413): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.372020] ---[ end trace 0000000000000000 ]---
<6> [178.372721] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.372842] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.372852] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.373053] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.373157] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.373262] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.373463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.373552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.373644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.373751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.373837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.373923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.374008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.374095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.374178] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.374272] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.374390] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.375395] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.385720] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.385964] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.386956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.387035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.387113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.387190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.387267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.387342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.387419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.387495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.387569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.387645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.387728] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.387806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.387883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.387958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.388034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.388107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.388181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.388256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.388333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.388417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.388500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.388581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.388662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.388749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.388828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.388907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.388985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.389064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.389144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.389223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.389305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.389386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.389471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.389554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.389637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.389722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.389802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.389879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.389961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.390039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.390119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.390198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.390276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.390353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.390434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.390510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.390590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.390670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.390759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.390837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.390915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.390995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.391077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.391158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.391236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.391315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.391391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.391467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.391546] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.391657] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.391661] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54519, lrc_seqno=54519, guc_id=0, flags=0x73 in no process [-1]
<7> [178.391664] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.392577] ------------[ cut here ]------------
<4> [178.392578] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.392579] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.392651] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.392721] efi_pstore nfnetlink autofs4
<4> [178.392725] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.392728] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.392729] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.392731] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.392736] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.392807] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.392809] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.392811] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.392813] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.392814] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.392815] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.392817] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.392818] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.392819] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.392821] CR2: 000060dda491f42c CR3: 0000000114e1a005 CR4: 0000000000f72ef0
<4> [178.392822] PKRU: 55555554
<4> [178.392824] Call Trace:
<4> [178.392825] <TASK>
<4> [178.392829] ? lock_sync+0x100/0x100
<4> [178.392834] ? lock_release+0xd0/0x2b0
<4> [178.392839] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.392844] process_one_work+0x239/0x760
<4> [178.392851] worker_thread+0x200/0x3f0
<4> [178.392854] ? __pfx_worker_thread+0x10/0x10
<4> [178.392856] kthread+0x10d/0x150
<4> [178.392859] ? __pfx_kthread+0x10/0x10
<4> [178.392863] ret_from_fork+0x3d4/0x480
<4> [178.392865] ? __pfx_kthread+0x10/0x10
<4> [178.392868] ret_from_fork_asm+0x1a/0x30
<4> [178.392875] </TASK>
<4> [178.392877] irq event stamp: 2470549
<4> [178.392878] hardirqs last enabled at (2470555): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.392881] hardirqs last disabled at (2470560): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.392883] softirqs last enabled at (2470296): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.392885] softirqs last disabled at (2470289): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.392888] ---[ end trace 0000000000000000 ]---
<5> [178.393694] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54574, lrc_seqno=54574, guc_id=0, flags=0x73 in no process [-1]
<7> [178.393701] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [178.393705] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [178.393819] ------------[ cut here ]------------
<4> [178.393821] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [178.393824] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [178.393824] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:39/3057
<4> [178.393942] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.394033] efi_pstore nfnetlink autofs4
<4> [178.394039] CPU: 12 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.394043] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.394044] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.394046] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.394054] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.394168] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.394170] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.394173] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.394175] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.394177] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.394179] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.394181] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.394183] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [178.394185] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.394187] CR2: 0000768ff00973c8 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [178.394189] PKRU: 55555554
<4> [178.394191] Call Trace:
<4> [178.394193] <TASK>
<4> [178.394199] ? lock_sync+0x100/0x100
<4> [178.394206] ? lock_release+0xd0/0x2b0
<4> [178.394215] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.394224] process_one_work+0x239/0x760
<4> [178.394234] worker_thread+0x200/0x3f0
<4> [178.394239] ? __pfx_worker_thread+0x10/0x10
<4> [178.394242] kthread+0x10d/0x150
<4> [178.394246] ? __pfx_kthread+0x10/0x10
<4> [178.394251] ret_from_fork+0x3d4/0x480
<4> [178.394254] ? __pfx_kthread+0x10/0x10
<4> [178.394259] ret_from_fork_asm+0x1a/0x30
<4> [178.394270] </TASK>
<4> [178.394272] irq event stamp: 169489
<4> [178.394273] hardirqs last enabled at (169495): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.394277] hardirqs last disabled at (169500): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.394280] softirqs last enabled at (168728): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.394284] softirqs last disabled at (168723): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.394287] ---[ end trace 0000000000000000 ]---
<6> [178.394289] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.394403] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.394411] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.394502] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.394706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.394808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.394906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.394995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.395083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.395167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.395253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.395341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.395427] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.395524] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.395641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.396647] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.406718] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.406967] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.408068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.408149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.408227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.408305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.408381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.408457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.408533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.408608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.408684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.408771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.408848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.408924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.408999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.409074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.409149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.409223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.409296] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.409372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.409449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.409533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.409614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.409694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.409786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.409864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.409942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.410020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.410098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.410177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.410258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.410337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.410417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.410497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.410583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.410669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.410763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.410846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.410927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.411004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.411082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.411157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.411237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.411313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.411391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.411471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.411555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.411633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.411715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.411793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.411872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.411950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.412027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.412106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.412183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.412263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.412344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.412423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.412501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.412578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.412662] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.412780] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.412784] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54574, lrc_seqno=54574, guc_id=0, flags=0x73 in no process [-1]
<7> [178.412786] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.412844] ------------[ cut here ]------------
<4> [178.412845] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.412847] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.412918] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.412982] efi_pstore nfnetlink autofs4
<4> [178.412986] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.412989] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.412990] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.412991] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.412996] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.413066] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.413068] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.413070] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.413072] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.413073] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.413074] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.413075] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.413077] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.413078] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.413080] CR2: 000060dda491f42c CR3: 0000000114e1a005 CR4: 0000000000f72ef0
<4> [178.413081] PKRU: 55555554
<4> [178.413082] Call Trace:
<4> [178.413083] <TASK>
<4> [178.413087] ? lock_sync+0x100/0x100
<4> [178.413092] ? lock_release+0xd0/0x2b0
<4> [178.413097] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.413103] process_one_work+0x239/0x760
<4> [178.413109] worker_thread+0x200/0x3f0
<4> [178.413111] ? __pfx_worker_thread+0x10/0x10
<4> [178.413114] kthread+0x10d/0x150
<4> [178.413117] ? __pfx_kthread+0x10/0x10
<4> [178.413120] ret_from_fork+0x3d4/0x480
<4> [178.413122] ? __pfx_kthread+0x10/0x10
<4> [178.413126] ret_from_fork_asm+0x1a/0x30
<4> [178.413133] </TASK>
<4> [178.413134] irq event stamp: 2475861
<4> [178.413135] hardirqs last enabled at (2475867): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.413138] hardirqs last disabled at (2475872): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.413140] softirqs last enabled at (2474758): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.413143] softirqs last disabled at (2474749): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.413145] ---[ end trace 0000000000000000 ]---
<6> [178.413147] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.413215] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.413221] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.413391] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.413593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.413677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.413776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.413863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.413954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.414041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.414128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.414215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.414296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.414390] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.414507] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.415512] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.426065] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.426307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.427322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.427394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.427464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.427534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.427602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.427672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.427749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.427824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.427899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.427974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.428048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.428123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.428198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.428273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.428347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.428425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.428502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.428578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.428653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.428742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.428824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.428905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.428986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.429066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.429147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.429226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.429304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.429386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.429469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.429550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.429631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.429714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.429798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.429882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.429966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.430047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.430127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.430205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.430281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.430357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.430437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.430515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.430591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.430667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.430758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.430840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.430920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.430998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.431076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.431152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.431229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.431313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.431394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.431473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.431551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.431629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.431704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.431795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.431874] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.431983] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.431987] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54574, lrc_seqno=54574, guc_id=0, flags=0x73 in no process [-1]
<7> [178.431989] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.432048] ------------[ cut here ]------------
<4> [178.432049] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.432050] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.432122] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.432185] efi_pstore nfnetlink autofs4
<4> [178.432189] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.432192] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.432193] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.432194] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.432199] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.432270] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.432271] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.432274] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.432275] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.432276] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.432277] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.432279] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.432280] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.432281] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.432283] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.432284] PKRU: 55555554
<4> [178.432285] Call Trace:
<4> [178.432286] <TASK>
<4> [178.432290] ? lock_sync+0x100/0x100
<4> [178.432295] ? lock_release+0xd0/0x2b0
<4> [178.432300] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.432306] process_one_work+0x239/0x760
<4> [178.432311] worker_thread+0x200/0x3f0
<4> [178.432314] ? __pfx_worker_thread+0x10/0x10
<4> [178.432317] kthread+0x10d/0x150
<4> [178.432319] ? __pfx_kthread+0x10/0x10
<4> [178.432323] ret_from_fork+0x3d4/0x480
<4> [178.432325] ? __pfx_kthread+0x10/0x10
<4> [178.432328] ret_from_fork_asm+0x1a/0x30
<4> [178.432336] </TASK>
<4> [178.432337] irq event stamp: 2478985
<4> [178.432338] hardirqs last enabled at (2478991): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.432341] hardirqs last disabled at (2478996): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.432343] softirqs last enabled at (2478136): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.432345] softirqs last disabled at (2478125): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.432347] ---[ end trace 0000000000000000 ]---
<5> [178.433576] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54629, lrc_seqno=54629, guc_id=0, flags=0x73 in no process [-1]
<7> [178.433579] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.433640] ------------[ cut here ]------------
<4> [178.433642] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.433643] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.433717] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.433780] efi_pstore nfnetlink autofs4
<4> [178.433784] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.433787] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.433788] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.433789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.433794] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.433862] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.433863] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.433865] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.433866] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.433868] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.433869] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.433870] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.433871] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.433873] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.433874] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.433875] PKRU: 55555554
<4> [178.433876] Call Trace:
<4> [178.433877] <TASK>
<4> [178.433881] ? lock_sync+0x100/0x100
<4> [178.433885] ? lock_release+0xd0/0x2b0
<4> [178.433891] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.433896] process_one_work+0x239/0x760
<4> [178.433902] worker_thread+0x200/0x3f0
<4> [178.433905] ? __pfx_worker_thread+0x10/0x10
<4> [178.433907] kthread+0x10d/0x150
<4> [178.433910] ? __pfx_kthread+0x10/0x10
<4> [178.433913] ret_from_fork+0x3d4/0x480
<4> [178.433915] ? __pfx_kthread+0x10/0x10
<4> [178.433919] ret_from_fork_asm+0x1a/0x30
<4> [178.433926] </TASK>
<4> [178.433927] irq event stamp: 2480881
<4> [178.433928] hardirqs last enabled at (2480887): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.433931] hardirqs last disabled at (2480892): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.433933] softirqs last enabled at (2480138): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.433935] softirqs last disabled at (2480131): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.433937] ---[ end trace 0000000000000000 ]---
<7> [178.433950] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.434055] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.434151] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.434221] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.434227] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.434276] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.434483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.434576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.434671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.434787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.434896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.434986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.435075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.435163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.435248] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.435345] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.435460] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.436475] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.446715] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.446964] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.447994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.448074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.448152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.448230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.448305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.448380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.448455] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.448529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.448602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.448676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.448765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.448854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.448928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.449002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.449076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.449155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.449232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.449308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.449383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.449465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.449546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.449626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.449712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.449805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.449884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.449965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.450045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.450127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.450208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.450287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.450367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.450449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.450538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.450625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.450716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.450821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.450902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.450981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.451060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.451136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.451217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.451300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.451382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.451460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.451542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.451620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.451698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.451789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.451883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.451965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.452043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.452124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.452202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.452279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.452356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.452434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.452510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.452586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.452665] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.452791] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.452798] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54629, lrc_seqno=54629, guc_id=0, flags=0x73 in no process [-1]
<7> [178.452802] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.452870] ------------[ cut here ]------------
<4> [178.452871] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.452873] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.452945] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.453009] efi_pstore nfnetlink autofs4
<4> [178.453014] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.453017] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.453018] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.453020] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.453025] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.453096] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.453097] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.453100] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.453101] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.453102] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.453104] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.453105] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.453106] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.453108] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.453109] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.453111] PKRU: 55555554
<4> [178.453112] Call Trace:
<4> [178.453113] <TASK>
<4> [178.453117] ? lock_sync+0x100/0x100
<4> [178.453122] ? lock_release+0xd0/0x2b0
<4> [178.453127] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.453133] process_one_work+0x239/0x760
<4> [178.453139] worker_thread+0x200/0x3f0
<4> [178.453142] ? __pfx_worker_thread+0x10/0x10
<4> [178.453144] kthread+0x10d/0x150
<4> [178.453147] ? __pfx_kthread+0x10/0x10
<4> [178.453151] ret_from_fork+0x3d4/0x480
<4> [178.453153] ? __pfx_kthread+0x10/0x10
<4> [178.453156] ret_from_fork_asm+0x1a/0x30
<4> [178.453163] </TASK>
<4> [178.453164] irq event stamp: 561337
<4> [178.453165] hardirqs last enabled at (561343): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.453168] hardirqs last disabled at (561348): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.453171] softirqs last enabled at (560422): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.453173] softirqs last disabled at (560415): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.453175] ---[ end trace 0000000000000000 ]---
<6> [178.453177] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.453247] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.453255] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.454502] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.454608] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.454732] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.454993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.455131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.455271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.455417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.455518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.455610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.455699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.455825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.455911] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.456007] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.456125] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.457153] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.467712] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.467959] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.468974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.469058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.469136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.469214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.469291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.469369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.469449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.469528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.469606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.469682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.469775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.469860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.469935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.470009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.470084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.470158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.470232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.470306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.470380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.470462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.470545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.470626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.470716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.470804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.470884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.470961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.471038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.471120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.471205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.471286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.471367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.471447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.471532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.471617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.471701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.471802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.471893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.471971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.472048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.472125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.472207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.472290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.472372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.472452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.472534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.472613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.472692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.472788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.472878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.472954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.473031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.473112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.473189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.473268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.473350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.473432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.473512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.473590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.473672] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.473805] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.473810] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54629, lrc_seqno=54629, guc_id=0, flags=0x73 in no process [-1]
<7> [178.473813] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.473871] ------------[ cut here ]------------
<4> [178.473873] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.473874] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [178.473946] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.474011] efi_pstore nfnetlink autofs4
<4> [178.474016] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.474019] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.474020] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.474022] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.474027] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.474096] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.474098] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.474100] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.474102] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.474103] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.474104] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.474105] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.474107] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.474108] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.474110] CR2: 000077a5bc1e62f4 CR3: 0000000115508001 CR4: 0000000000f72ef0
<4> [178.474111] PKRU: 55555554
<4> [178.474113] Call Trace:
<4> [178.474114] <TASK>
<4> [178.474118] ? lock_sync+0x100/0x100
<4> [178.474122] ? lock_release+0xd0/0x2b0
<4> [178.474128] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.474133] process_one_work+0x239/0x760
<4> [178.474140] worker_thread+0x200/0x3f0
<4> [178.474143] ? __pfx_worker_thread+0x10/0x10
<4> [178.474145] kthread+0x10d/0x150
<4> [178.474148] ? __pfx_kthread+0x10/0x10
<4> [178.474152] ret_from_fork+0x3d4/0x480
<4> [178.474154] ? __pfx_kthread+0x10/0x10
<4> [178.474157] ret_from_fork_asm+0x1a/0x30
<4> [178.474164] </TASK>
<4> [178.474166] irq event stamp: 173253
<4> [178.474167] hardirqs last enabled at (173259): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.474170] hardirqs last disabled at (173264): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.474172] softirqs last enabled at (172418): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.474174] softirqs last disabled at (172411): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.474177] ---[ end trace 0000000000000000 ]---
<5> [178.475660] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54684, lrc_seqno=54684, guc_id=0, flags=0x73 in no process [-1]
<7> [178.475663] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.475740] ------------[ cut here ]------------
<4> [178.475742] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.475744] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [178.475825] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.475888] efi_pstore nfnetlink autofs4
<4> [178.475893] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.475896] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.475897] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.475898] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.475903] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.475973] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.475975] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.475977] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.475978] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.475980] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.475981] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.475982] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.475983] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.475985] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.475986] CR2: 000077a5bc1e62f4 CR3: 0000000115508001 CR4: 0000000000f72ef0
<4> [178.475988] PKRU: 55555554
<4> [178.475989] Call Trace:
<4> [178.475990] <TASK>
<4> [178.475994] ? lock_sync+0x100/0x100
<4> [178.475998] ? lock_release+0xd0/0x2b0
<4> [178.476003] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.476009] process_one_work+0x239/0x760
<4> [178.476015] worker_thread+0x200/0x3f0
<4> [178.476018] ? __pfx_worker_thread+0x10/0x10
<4> [178.476020] kthread+0x10d/0x150
<4> [178.476023] ? __pfx_kthread+0x10/0x10
<4> [178.476027] ret_from_fork+0x3d4/0x480
<4> [178.476029] ? __pfx_kthread+0x10/0x10
<4> [178.476032] ret_from_fork_asm+0x1a/0x30
<4> [178.476039] </TASK>
<4> [178.476040] irq event stamp: 176021
<4> [178.476042] hardirqs last enabled at (176027): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.476044] hardirqs last disabled at (176032): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.476046] softirqs last enabled at (172418): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.476049] softirqs last disabled at (172411): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.476051] ---[ end trace 0000000000000000 ]---
<6> [178.476053] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.476123] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.476129] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.476337] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.476441] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.476548] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.476759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.476860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.476953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.477043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.477131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.477218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.477305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.477392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.477473] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.477568] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.477685] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.478711] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.488710] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.488961] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.489969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.490050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.490127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.490205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.490280] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.490355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.490429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.490504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.490579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.490654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.490743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.490829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.490905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.490980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.491055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.491128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.491202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.491275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.491348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.491430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.491512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.491593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.491676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.491768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.491857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.491936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.492013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.492093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.492172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.492251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.492330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.492410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.492499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.492587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.492673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.492770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.492865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.492946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.493027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.493105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.493186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.493265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.493343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.493420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.493501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.493579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.493659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.493752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.493844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.493921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.493997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.494076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.494154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.494234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.494316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.494398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.494477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.494555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.494635] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.494760] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.494764] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54684, lrc_seqno=54684, guc_id=0, flags=0x73 in no process [-1]
<7> [178.494767] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.494827] ------------[ cut here ]------------
<4> [178.494828] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.494829] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [178.494902] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.494966] efi_pstore nfnetlink autofs4
<4> [178.494971] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.494974] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.494975] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.494976] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.494981] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.495053] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.495055] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.495057] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.495058] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.495060] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.495061] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.495062] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.495063] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.495065] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.495066] CR2: 000077a5bc1e62f4 CR3: 0000000115508001 CR4: 0000000000f72ef0
<4> [178.495068] PKRU: 55555554
<4> [178.495069] Call Trace:
<4> [178.495070] <TASK>
<4> [178.495074] ? lock_sync+0x100/0x100
<4> [178.495078] ? lock_release+0xd0/0x2b0
<4> [178.495084] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.495090] process_one_work+0x239/0x760
<4> [178.495096] worker_thread+0x200/0x3f0
<4> [178.495098] ? __pfx_worker_thread+0x10/0x10
<4> [178.495101] kthread+0x10d/0x150
<4> [178.495104] ? __pfx_kthread+0x10/0x10
<4> [178.495107] ret_from_fork+0x3d4/0x480
<4> [178.495110] ? __pfx_kthread+0x10/0x10
<4> [178.495113] ret_from_fork_asm+0x1a/0x30
<4> [178.495120] </TASK>
<4> [178.495121] irq event stamp: 179109
<4> [178.495122] hardirqs last enabled at (179115): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.495125] hardirqs last disabled at (179120): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.495127] softirqs last enabled at (178012): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.495130] softirqs last disabled at (178007): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.495132] ---[ end trace 0000000000000000 ]---
<6> [178.495134] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.495203] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.495209] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.496446] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.496552] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.496658] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.496894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.496987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.497082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.497173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.497262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.497347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.497433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.497519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.497602] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.497696] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.497831] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.498860] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.508708] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.508956] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.509966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.510047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.510123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.510201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.510278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.510355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.510432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.510511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.510588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.510664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.510751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.510846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.510921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.510995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.511069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.511145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.511222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.511300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.511375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.511458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.511540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.511621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.511709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.511806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.511885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.511964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.512042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.512126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.512210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.512291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.512372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.512454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.512543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.512631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.512731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.512827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.512910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.512989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.513068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.513146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.513232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.513314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.513395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.513471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.513554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.513633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.513717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.513816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.513895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.513970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.514047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.514130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.514212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.514292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.514371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.514450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.514528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.514605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.514686] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.514817] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.514824] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54684, lrc_seqno=54684, guc_id=0, flags=0x73 in no process [-1]
<7> [178.514828] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.514890] ------------[ cut here ]------------
<4> [178.514891] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.514893] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.514965] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.515029] efi_pstore nfnetlink autofs4
<4> [178.515034] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.515037] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.515038] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.515040] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.515045] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.515116] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.515118] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.515120] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.515121] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.515123] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.515124] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.515125] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.515127] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.515128] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.515130] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.515131] PKRU: 55555554
<4> [178.515132] Call Trace:
<4> [178.515133] <TASK>
<4> [178.515137] ? lock_sync+0x100/0x100
<4> [178.515142] ? lock_release+0xd0/0x2b0
<4> [178.515147] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.515153] process_one_work+0x239/0x760
<4> [178.515159] worker_thread+0x200/0x3f0
<4> [178.515162] ? __pfx_worker_thread+0x10/0x10
<4> [178.515164] kthread+0x10d/0x150
<4> [178.515167] ? __pfx_kthread+0x10/0x10
<4> [178.515170] ret_from_fork+0x3d4/0x480
<4> [178.515172] ? __pfx_kthread+0x10/0x10
<4> [178.515176] ret_from_fork_asm+0x1a/0x30
<4> [178.515183] </TASK>
<4> [178.515184] irq event stamp: 567419
<4> [178.515185] hardirqs last enabled at (567425): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.515188] hardirqs last disabled at (567430): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.515190] softirqs last enabled at (566498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.515193] softirqs last disabled at (566491): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.515195] ---[ end trace 0000000000000000 ]---
<5> [178.516990] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54739, lrc_seqno=54739, guc_id=0, flags=0x73 in no process [-1]
<7> [178.516993] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.517053] ------------[ cut here ]------------
<4> [178.517054] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.517056] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.517127] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.517194] efi_pstore nfnetlink autofs4
<4> [178.517199] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.517202] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.517203] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.517205] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.517210] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.517279] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.517281] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.517283] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.517285] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.517286] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.517288] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.517289] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.517291] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.517293] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.517294] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.517296] PKRU: 55555554
<4> [178.517297] Call Trace:
<4> [178.517299] <TASK>
<4> [178.517302] ? lock_sync+0x100/0x100
<4> [178.517307] ? lock_release+0xd0/0x2b0
<4> [178.517312] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.517318] process_one_work+0x239/0x760
<4> [178.517324] worker_thread+0x200/0x3f0
<4> [178.517327] ? __pfx_worker_thread+0x10/0x10
<4> [178.517330] kthread+0x10d/0x150
<4> [178.517333] ? __pfx_kthread+0x10/0x10
<4> [178.517337] ret_from_fork+0x3d4/0x480
<4> [178.517339] ? __pfx_kthread+0x10/0x10
<4> [178.517342] ret_from_fork_asm+0x1a/0x30
<4> [178.517350] </TASK>
<4> [178.517351] irq event stamp: 570925
<4> [178.517352] hardirqs last enabled at (570931): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.517355] hardirqs last disabled at (570936): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.517357] softirqs last enabled at (566498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.517360] softirqs last disabled at (566491): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.517362] ---[ end trace 0000000000000000 ]---
<6> [178.517364] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.517434] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.517440] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.517674] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.517789] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.517946] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.518159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.518248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.518342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.518430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.518516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.518600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.518685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.518789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.518886] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.518980] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.519095] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.520109] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.530706] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.530958] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.531942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.532016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.532087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.532158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.532228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.532300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.532371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.532440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.532508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.532577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.532646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.532720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.532809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.532883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.532958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.533032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.533110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.533187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.533263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.533345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.533426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.533506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.533587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.533667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.533758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.533851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.533929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.534012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.534096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.534178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.534259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.534340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.534426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.534511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.534595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.534676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.534767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.534867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.534945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.535025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.535107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.535187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.535265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.535341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.535423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.535500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.535579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.535655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.535744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.535837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.535918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.536001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.536081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.536160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.536242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.536326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.536407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.536485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.536566] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.536678] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.536682] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54739, lrc_seqno=54739, guc_id=0, flags=0x73 in no process [-1]
<7> [178.536685] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.537626] ------------[ cut here ]------------
<4> [178.537627] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.537629] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.537706] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.537790] efi_pstore nfnetlink autofs4
<4> [178.537795] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.537798] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.537799] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.537800] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.537806] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.537876] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.537877] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.537880] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.537881] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.537883] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.537884] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.537885] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.537886] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.537888] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.537889] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.537891] PKRU: 55555554
<4> [178.537892] Call Trace:
<4> [178.537893] <TASK>
<4> [178.537897] ? lock_sync+0x100/0x100
<4> [178.537901] ? lock_release+0xd0/0x2b0
<4> [178.537907] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.537912] process_one_work+0x239/0x760
<4> [178.537919] worker_thread+0x200/0x3f0
<4> [178.537922] ? __pfx_worker_thread+0x10/0x10
<4> [178.537924] kthread+0x10d/0x150
<4> [178.537927] ? __pfx_kthread+0x10/0x10
<4> [178.537930] ret_from_fork+0x3d4/0x480
<4> [178.537932] ? __pfx_kthread+0x10/0x10
<4> [178.537936] ret_from_fork_asm+0x1a/0x30
<4> [178.537943] </TASK>
<4> [178.537944] irq event stamp: 574025
<4> [178.537945] hardirqs last enabled at (574031): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.537948] hardirqs last disabled at (574036): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.537950] softirqs last enabled at (573024): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.537953] softirqs last disabled at (573017): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.537955] ---[ end trace 0000000000000000 ]---
<6> [178.537957] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.538029] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.538036] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.538452] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.538556] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.538660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.538883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.538973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.539065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.539153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.539241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.539329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.539420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.539510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.539594] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.539689] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.539825] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.540856] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.550703] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.550950] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.551999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.552082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.552162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.552242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.552320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.552397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.552474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.552550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.552624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.552704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.552796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.552875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.552951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.553027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.553102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.553177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.553252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.553326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.553402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.553486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.553569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.553649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.553745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.553839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.553919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.553996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.554072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.554152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.554232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.554310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.554389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.554467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.554552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.554640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.554740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.554846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.554929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.555008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.555086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.555163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.555243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.555321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.555402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.555482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.555565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.555645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.555733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.555827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.555910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.555988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.556068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.556152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.556233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.556311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.556389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.556467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.556545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.556625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.556712] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.556844] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.556848] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54739, lrc_seqno=54739, guc_id=0, flags=0x73 in no process [-1]
<7> [178.556850] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.556909] ------------[ cut here ]------------
<4> [178.556910] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.556911] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.556984] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.557048] efi_pstore nfnetlink autofs4
<4> [178.557053] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.557055] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.557057] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.557058] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.557063] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.557133] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.557135] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.557137] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.557139] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.557140] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.557141] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.557143] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.557144] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.557146] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.557147] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.557149] PKRU: 55555554
<4> [178.557150] Call Trace:
<4> [178.557151] <TASK>
<4> [178.557155] ? lock_sync+0x100/0x100
<4> [178.557159] ? lock_release+0xd0/0x2b0
<4> [178.557165] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.557170] process_one_work+0x239/0x760
<4> [178.557176] worker_thread+0x200/0x3f0
<4> [178.557179] ? __pfx_worker_thread+0x10/0x10
<4> [178.557181] kthread+0x10d/0x150
<4> [178.557184] ? __pfx_kthread+0x10/0x10
<4> [178.557188] ret_from_fork+0x3d4/0x480
<4> [178.557190] ? __pfx_kthread+0x10/0x10
<4> [178.557193] ret_from_fork_asm+0x1a/0x30
<4> [178.557201] </TASK>
<4> [178.557202] irq event stamp: 577133
<4> [178.557203] hardirqs last enabled at (577139): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.557206] hardirqs last disabled at (577144): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.557208] softirqs last enabled at (576120): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.557210] softirqs last disabled at (576113): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.557212] ---[ end trace 0000000000000000 ]---
<5> [178.558501] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54794, lrc_seqno=54794, guc_id=0, flags=0x73 in no process [-1]
<7> [178.558504] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.558565] ------------[ cut here ]------------
<4> [178.558566] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.558568] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.558639] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.558716] efi_pstore nfnetlink autofs4
<4> [178.558723] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.558727] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.558729] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.558732] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.558739] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.558813] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.558814] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.558816] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.558818] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.558819] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.558820] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.558821] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.558823] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.558824] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.558826] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.558827] PKRU: 55555554
<4> [178.558828] Call Trace:
<4> [178.558829] <TASK>
<4> [178.558833] ? lock_sync+0x100/0x100
<4> [178.558837] ? lock_release+0xd0/0x2b0
<4> [178.558842] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.558848] process_one_work+0x239/0x760
<4> [178.558854] worker_thread+0x200/0x3f0
<4> [178.558857] ? __pfx_worker_thread+0x10/0x10
<4> [178.558859] kthread+0x10d/0x150
<4> [178.558862] ? __pfx_kthread+0x10/0x10
<4> [178.558865] ret_from_fork+0x3d4/0x480
<4> [178.558867] ? __pfx_kthread+0x10/0x10
<4> [178.558871] ret_from_fork_asm+0x1a/0x30
<4> [178.558878] </TASK>
<4> [178.558879] irq event stamp: 579297
<4> [178.558880] hardirqs last enabled at (579303): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.558883] hardirqs last disabled at (579308): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.558885] softirqs last enabled at (578988): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.558887] softirqs last disabled at (578977): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.558889] ---[ end trace 0000000000000000 ]---
<7> [178.558903] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.559008] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.559108] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.559181] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.559187] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.559236] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.559444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.559536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.559633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.559748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.559847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.559934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.560021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.560109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.560193] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.560289] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.560408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.561422] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.571700] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.571950] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.572981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.573064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.573141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.573220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.573298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.573375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.573451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.573530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.573607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.573683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.573779] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.573862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.573938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.574014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.574089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.574166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.574244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.574322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.574400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.574483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.574565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.574645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.574739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.574828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.574907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.574984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.575061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.575144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.575230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.575312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.575393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.575472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.575556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.575641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.575737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.575825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.575905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.575984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.576061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.576137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.576222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.576304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.576384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.576462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.576544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.576622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.576707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.576783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.576861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.576937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.577013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.577092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.577168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.577246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.577325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.577406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.577484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.577561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.577640] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.577762] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.577766] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54794, lrc_seqno=54794, guc_id=0, flags=0x73 in no process [-1]
<7> [178.577768] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.577827] ------------[ cut here ]------------
<4> [178.577828] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.577829] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:39/3057
<4> [178.577901] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.577965] efi_pstore nfnetlink autofs4
<4> [178.577969] CPU: 3 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.577972] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.577974] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.577975] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.577980] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.578051] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.578052] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.578055] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.578056] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.578058] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.578059] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.578060] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.578061] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.578063] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.578064] CR2: 000077a5bc1e62f4 CR3: 0000000114e1a006 CR4: 0000000000f72ef0
<4> [178.578066] PKRU: 55555554
<4> [178.578067] Call Trace:
<4> [178.578068] <TASK>
<4> [178.578072] ? lock_sync+0x100/0x100
<4> [178.578077] ? lock_release+0xd0/0x2b0
<4> [178.578082] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.578088] process_one_work+0x239/0x760
<4> [178.578094] worker_thread+0x200/0x3f0
<4> [178.578097] ? __pfx_worker_thread+0x10/0x10
<4> [178.578099] kthread+0x10d/0x150
<4> [178.578102] ? __pfx_kthread+0x10/0x10
<4> [178.578106] ret_from_fork+0x3d4/0x480
<4> [178.578108] ? __pfx_kthread+0x10/0x10
<4> [178.578112] ret_from_fork_asm+0x1a/0x30
<4> [178.578119] </TASK>
<4> [178.578120] irq event stamp: 184469
<4> [178.578121] hardirqs last enabled at (184475): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.578124] hardirqs last disabled at (184480): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.578126] softirqs last enabled at (183634): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.578128] softirqs last disabled at (183619): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.578131] ---[ end trace 0000000000000000 ]---
<6> [178.578132] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.578203] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.578209] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.579439] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.579544] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.579659] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.579891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.579981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.580075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.580165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.580253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.580341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.580431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.580522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.580607] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.580710] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.580840] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.581859] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.591698] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.591946] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.592950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.593030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.593107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.593185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.593261] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.593338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.593414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.593490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.593564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.593639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.593724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.593825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.593903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.593979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.594056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.594131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.594205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.594278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.594353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.594435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.594516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.594595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.594676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.594783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.594881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.594961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.595040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.595122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.595203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.595283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.595362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.595442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.595526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.595612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.595705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.595807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.595891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.595971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.596051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.596130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.596211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.596289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.596368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.596446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.596530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.596611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.596697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.596798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.596879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.596956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.597035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.597118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.597199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.597278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.597356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.597435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.597511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.597587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.597670] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.597804] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.597810] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54794, lrc_seqno=54794, guc_id=0, flags=0x73 in no process [-1]
<7> [178.597814] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.597880] ------------[ cut here ]------------
<4> [178.597881] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.597882] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:45/3063
<4> [178.597955] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.598019] efi_pstore nfnetlink autofs4
<4> [178.598024] CPU: 5 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.598027] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.598028] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.598029] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.598035] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.598104] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.598106] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.598108] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.598109] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.598111] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.598112] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.598113] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.598115] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000
<4> [178.598116] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.598117] CR2: 0000758009e44e78 CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.598119] PKRU: 55555554
<4> [178.598120] Call Trace:
<4> [178.598121] <TASK>
<4> [178.598125] ? lock_sync+0x100/0x100
<4> [178.598129] ? lock_release+0xd0/0x2b0
<4> [178.598135] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.598140] process_one_work+0x239/0x760
<4> [178.598147] worker_thread+0x200/0x3f0
<4> [178.598149] ? __pfx_worker_thread+0x10/0x10
<4> [178.598152] kthread+0x10d/0x150
<4> [178.598155] ? __pfx_kthread+0x10/0x10
<4> [178.598158] ret_from_fork+0x3d4/0x480
<4> [178.598160] ? __pfx_kthread+0x10/0x10
<4> [178.598164] ret_from_fork_asm+0x1a/0x30
<4> [178.598171] </TASK>
<4> [178.598172] irq event stamp: 582743
<4> [178.598173] hardirqs last enabled at (582749): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.598176] hardirqs last disabled at (582754): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.598178] softirqs last enabled at (581908): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.598181] softirqs last disabled at (581897): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.598183] ---[ end trace 0000000000000000 ]---
<5> [178.599462] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54849, lrc_seqno=54849, guc_id=0, flags=0x73 in no process [-1]
<7> [178.599464] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.599469] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.599577] ------------[ cut here ]------------
<4> [178.599579] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [178.599572] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [178.599581] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:9/209
<4> [178.599707] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.599798] efi_pstore nfnetlink autofs4
<4> [178.599804] CPU: 12 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.599808] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.599810] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.599812] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.599820] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.599933] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.599935] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.599939] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.599941] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.599943] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.599944] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.599946] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.599948] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [178.599950] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.599952] CR2: 0000768ff00973c8 CR3: 000000000344c002 CR4: 0000000000f72ef0
<4> [178.599955] PKRU: 55555554
<4> [178.599956] Call Trace:
<4> [178.599958] <TASK>
<4> [178.599964] ? lock_sync+0x100/0x100
<4> [178.599971] ? lock_release+0xd0/0x2b0
<4> [178.599980] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.599989] process_one_work+0x239/0x760
<4> [178.599999] worker_thread+0x200/0x3f0
<4> [178.600003] ? __pfx_worker_thread+0x10/0x10
<4> [178.600007] kthread+0x10d/0x150
<4> [178.600011] ? __pfx_kthread+0x10/0x10
<4> [178.600016] ret_from_fork+0x3d4/0x480
<4> [178.600019] ? __pfx_kthread+0x10/0x10
<4> [178.600023] ret_from_fork_asm+0x1a/0x30
<4> [178.600036] </TASK>
<4> [178.600037] irq event stamp: 2486277
<4> [178.600039] hardirqs last enabled at (2486283): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.600043] hardirqs last disabled at (2486288): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.600046] softirqs last enabled at (2485534): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.600049] softirqs last disabled at (2485529): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.600052] ---[ end trace 0000000000000000 ]---
<6> [178.600055] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.600168] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.600179] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.600319] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.600580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.600728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.600874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.601011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.601146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.601280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.601414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.601551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.601682] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.601831] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.601997] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.603354] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.613699] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.614012] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.615345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.615466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.615585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.615715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.615837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.615958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.616078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.616199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.616320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.616440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.616560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.616681] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.616807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.616928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.617048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.617169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.617290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.617411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.617531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.617656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.617793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.617918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.618043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.618167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.618290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.618413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.618536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.618663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.618796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.618923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.619050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.619176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.619308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.619439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.619570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.619704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.619833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.619958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.620083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.620217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.620346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.620473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.620598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.620736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.620868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.620994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.621119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.621243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.621368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.621491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.621614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.621748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.621875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.621999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.622123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.622248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.622371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.622494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.622621] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.622786] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.622793] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54849, lrc_seqno=54849, guc_id=0, flags=0x73 in no process [-1]
<7> [178.622796] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.622898] ------------[ cut here ]------------
<4> [178.622900] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.622902] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#14: kworker/u64:45/3063
<4> [178.623015] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.623105] efi_pstore nfnetlink autofs4
<4> [178.623112] CPU: 14 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.623116] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.623117] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.623119] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.623127] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.623239] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.623241] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.623245] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.623247] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.623249] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.623250] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.623252] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.623254] FS: 0000000000000000(0000) GS:ffff8888db397000(0000) knlGS:0000000000000000
<4> [178.623256] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.623258] CR2: 000061025014c6b8 CR3: 000000012cae9004 CR4: 0000000000f72ef0
<4> [178.623261] PKRU: 55555554
<4> [178.623262] Call Trace:
<4> [178.623264] <TASK>
<4> [178.623270] ? lock_sync+0x100/0x100
<4> [178.623277] ? lock_release+0xd0/0x2b0
<4> [178.623286] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.623295] process_one_work+0x239/0x760
<4> [178.623305] worker_thread+0x200/0x3f0
<4> [178.623309] ? __pfx_worker_thread+0x10/0x10
<4> [178.623313] kthread+0x10d/0x150
<4> [178.623316] ? __pfx_kthread+0x10/0x10
<4> [178.623322] ret_from_fork+0x3d4/0x480
<4> [178.623324] ? __pfx_kthread+0x10/0x10
<4> [178.623329] ret_from_fork_asm+0x1a/0x30
<4> [178.623341] </TASK>
<4> [178.623343] irq event stamp: 588281
<4> [178.623344] hardirqs last enabled at (588287): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.623348] hardirqs last disabled at (588292): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.623352] softirqs last enabled at (587446): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.623355] softirqs last disabled at (587439): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.623358] ---[ end trace 0000000000000000 ]---
<6> [178.623360] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.623473] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.623482] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.625162] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.625329] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.625488] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.625753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.625889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.626028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.626160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.626292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.626423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.626555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.627118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.627217] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.627332] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.627449] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.628458] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.638692] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.638940] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.639930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.640012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.640091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.640169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.640246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.640320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.640395] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.640470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.640544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.640618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.640696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.640773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.640850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.640927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.641002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.641076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.641150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.641223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.641297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.641380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.641461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.641540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.641621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.641704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.641787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.641868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.641947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.642027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.642108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.642188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.642267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.642346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.642431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.642514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.642599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.642680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.642774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.642852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.642929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.643005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.643084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.643162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.643239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.643319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.643403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.643483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.643562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.643639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.643728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.643805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.643882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.643962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.644042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.644122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.644201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.644280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.644357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.644434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.644513] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.644626] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.644630] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54849, lrc_seqno=54849, guc_id=0, flags=0x73 in no process [-1]
<7> [178.644633] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.645552] ------------[ cut here ]------------
<4> [178.645553] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.645554] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:9/209
<4> [178.645626] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.645697] efi_pstore nfnetlink autofs4
<4> [178.645702] CPU: 8 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.645705] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.645706] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.645707] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.645713] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.645785] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.645786] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.645789] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.645790] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.645792] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.645793] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.645794] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.645795] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.645797] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.645798] CR2: 000060dda491f42c CR3: 0000000115508004 CR4: 0000000000f72ef0
<4> [178.645800] PKRU: 55555554
<4> [178.645801] Call Trace:
<4> [178.645802] <TASK>
<4> [178.645806] ? lock_sync+0x100/0x100
<4> [178.645811] ? lock_release+0xd0/0x2b0
<4> [178.645816] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.645822] process_one_work+0x239/0x760
<4> [178.645828] worker_thread+0x200/0x3f0
<4> [178.645831] ? __pfx_worker_thread+0x10/0x10
<4> [178.645833] kthread+0x10d/0x150
<4> [178.645836] ? __pfx_kthread+0x10/0x10
<4> [178.645840] ret_from_fork+0x3d4/0x480
<4> [178.645842] ? __pfx_kthread+0x10/0x10
<4> [178.645846] ret_from_fork_asm+0x1a/0x30
<4> [178.645853] </TASK>
<4> [178.645854] irq event stamp: 2489769
<4> [178.645855] hardirqs last enabled at (2489775): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.645858] hardirqs last disabled at (2489780): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.645860] softirqs last enabled at (2488854): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.645863] softirqs last disabled at (2488845): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.645865] ---[ end trace 0000000000000000 ]---
<7> [178.645879] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.645983] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [178.646679] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54904, lrc_seqno=54904, guc_id=0, flags=0x73 in no process [-1]
<7> [178.646699] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.646760] ------------[ cut here ]------------
<4> [178.646761] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.646762] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:45/3063
<4> [178.646834] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.646898] efi_pstore nfnetlink autofs4
<4> [178.646903] CPU: 8 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.646905] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.646907] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.646908] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.646913] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.646981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.646983] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.646985] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.646986] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.646987] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.646988] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.646990] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.646991] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.646992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.646994] CR2: 000060dda491f42c CR3: 0000000115508004 CR4: 0000000000f72ef0
<4> [178.646995] PKRU: 55555554
<4> [178.646996] Call Trace:
<4> [178.646997] <TASK>
<4> [178.647001] ? lock_sync+0x100/0x100
<4> [178.647005] ? lock_release+0xd0/0x2b0
<4> [178.647011] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.647016] process_one_work+0x239/0x760
<4> [178.647023] worker_thread+0x200/0x3f0
<4> [178.647025] ? __pfx_worker_thread+0x10/0x10
<4> [178.647028] kthread+0x10d/0x150
<4> [178.647031] ? __pfx_kthread+0x10/0x10
<4> [178.647034] ret_from_fork+0x3d4/0x480
<4> [178.647036] ? __pfx_kthread+0x10/0x10
<4> [178.647040] ret_from_fork_asm+0x1a/0x30
<4> [178.647047] </TASK>
<4> [178.647048] irq event stamp: 590497
<4> [178.647049] hardirqs last enabled at (590503): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.647052] hardirqs last disabled at (590508): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.647054] softirqs last enabled at (588642): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [178.647057] softirqs last disabled at (588640): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [178.647059] ---[ end trace 0000000000000000 ]---
<6> [178.647061] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.647129] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.647258] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.647305] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.647505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.647590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.647677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.647787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.647878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.647967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.648048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.648128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.648203] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.648292] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.648401] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.649398] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.659688] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.659931] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.661033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.661113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.661189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.661267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.661344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.661419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.661496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.661575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.661653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.661738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.661812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.661888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.661963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.662037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.662111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.662185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.662259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.662333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.662407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.662489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.662570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.662649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.662739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.662819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.662897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.662977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.663058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.663140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.663221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.663301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.663381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.663460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.663544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.663628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.663723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.663807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.663888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.663967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.664046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.664122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.664203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.664280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.664357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.664434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.664513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.664591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.664671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.664761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.664840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.664915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.664992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.665071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.665148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.665224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.665299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.665376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.665450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.665527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.665611] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.665727] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.665731] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54904, lrc_seqno=54904, guc_id=0, flags=0x73 in no process [-1]
<7> [178.665734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.665792] ------------[ cut here ]------------
<4> [178.665794] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.665795] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:45/3063
<4> [178.665867] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.665930] efi_pstore nfnetlink autofs4
<4> [178.665935] CPU: 8 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.665937] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.665939] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.665940] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.665945] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.666014] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.666016] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.666018] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.666020] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.666021] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.666022] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.666023] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.666025] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.666026] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.666028] CR2: 000060dda491f42c CR3: 0000000114e1a005 CR4: 0000000000f72ef0
<4> [178.666029] PKRU: 55555554
<4> [178.666030] Call Trace:
<4> [178.666031] <TASK>
<4> [178.666035] ? lock_sync+0x100/0x100
<4> [178.666040] ? lock_release+0xd0/0x2b0
<4> [178.666045] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.666051] process_one_work+0x239/0x760
<4> [178.666056] worker_thread+0x200/0x3f0
<4> [178.666059] ? __pfx_worker_thread+0x10/0x10
<4> [178.666062] kthread+0x10d/0x150
<4> [178.666064] ? __pfx_kthread+0x10/0x10
<4> [178.666068] ret_from_fork+0x3d4/0x480
<4> [178.666070] ? __pfx_kthread+0x10/0x10
<4> [178.666073] ret_from_fork_asm+0x1a/0x30
<4> [178.666081] </TASK>
<4> [178.666082] irq event stamp: 593665
<4> [178.666083] hardirqs last enabled at (593671): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.666086] hardirqs last disabled at (593676): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.666088] softirqs last enabled at (592562): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.666090] softirqs last disabled at (592553): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.666093] ---[ end trace 0000000000000000 ]---
<6> [178.666094] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.666164] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.666170] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.666575] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.666779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.666865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.666957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.667048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.667135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.667221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.667306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.667394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.667476] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.667571] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.667692] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.668700] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.679423] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.679665] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.680810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.680889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.680965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.681043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.681119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.681198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.681277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.681353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.681428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.681502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.681575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.681649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.681732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.681806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.681881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.681956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.682030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.682104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.682179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.682263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.682348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.682430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.682511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.682589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.682666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.682759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.682839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.682921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.683004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.683085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.683164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.683245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.683330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.683416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.683500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.683582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.683663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.683751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.683829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.683906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.683987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.684072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.684153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.684232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.684314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.684392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.684470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.684548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.684626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.684715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.684797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.684880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.684960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.685039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.685117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.685197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.685274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.685355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.685438] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.685548] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.685552] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54904, lrc_seqno=54904, guc_id=0, flags=0x73 in no process [-1]
<7> [178.685555] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.685614] ------------[ cut here ]------------
<4> [178.685616] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.685617] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:45/3063
<4> [178.685692] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.685756] efi_pstore nfnetlink autofs4
<4> [178.685760] CPU: 8 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.685763] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.685764] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.685765] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.685770] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.685840] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.685842] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.685844] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.685845] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.685847] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.685848] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.685849] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.685850] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.685852] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.685853] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.685854] PKRU: 55555554
<4> [178.685856] Call Trace:
<4> [178.685857] <TASK>
<4> [178.685860] ? lock_sync+0x100/0x100
<4> [178.685865] ? lock_release+0xd0/0x2b0
<4> [178.685870] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.685876] process_one_work+0x239/0x760
<4> [178.685882] worker_thread+0x200/0x3f0
<4> [178.685884] ? __pfx_worker_thread+0x10/0x10
<4> [178.685887] kthread+0x10d/0x150
<4> [178.685890] ? __pfx_kthread+0x10/0x10
<4> [178.685893] ret_from_fork+0x3d4/0x480
<4> [178.685895] ? __pfx_kthread+0x10/0x10
<4> [178.685899] ret_from_fork_asm+0x1a/0x30
<4> [178.685906] </TASK>
<4> [178.685907] irq event stamp: 596765
<4> [178.685908] hardirqs last enabled at (596771): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.685911] hardirqs last disabled at (596776): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.685913] softirqs last enabled at (595862): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.685916] softirqs last disabled at (595857): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.685918] ---[ end trace 0000000000000000 ]---
<7> [178.687432] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<5> [178.687559] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54959, lrc_seqno=54959, guc_id=0, flags=0x73 in no process [-1]
<7> [178.687552] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.687574] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.687645] ------------[ cut here ]------------
<4> [178.687646] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.687649] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.687739] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.687804] efi_pstore nfnetlink autofs4
<4> [178.687809] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.687812] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.687813] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.687814] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.687820] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.687896] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.687898] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.687900] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.687902] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.687903] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.687904] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.687905] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.687907] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.687908] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.687910] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [178.687911] PKRU: 55555554
<4> [178.687913] Call Trace:
<4> [178.687914] <TASK>
<4> [178.687918] ? lock_sync+0x100/0x100
<4> [178.687923] ? lock_release+0xd0/0x2b0
<4> [178.687928] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.687934] process_one_work+0x239/0x760
<4> [178.687940] worker_thread+0x200/0x3f0
<4> [178.687943] ? __pfx_worker_thread+0x10/0x10
<4> [178.687945] kthread+0x10d/0x150
<4> [178.687948] ? __pfx_kthread+0x10/0x10
<4> [178.687952] ret_from_fork+0x3d4/0x480
<4> [178.687954] ? __pfx_kthread+0x10/0x10
<4> [178.687957] ret_from_fork_asm+0x1a/0x30
<4> [178.687965] </TASK>
<4> [178.687966] irq event stamp: 2494877
<4> [178.687967] hardirqs last enabled at (2494883): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.687970] hardirqs last disabled at (2494888): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.687972] softirqs last enabled at (2492140): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [178.687975] softirqs last disabled at (2492138): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [178.687977] ---[ end trace 0000000000000000 ]---
<6> [178.687979] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.688054] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.688062] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.688121] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.688324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.688412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.688505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.688592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.688681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.688765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.688850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.688935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.689015] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.689110] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.689227] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.690247] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.700683] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.700932] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.701946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.702025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.702103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.702180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.702256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.702332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.702407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.702482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.702559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.702636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.702722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.702798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.702873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.702947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.703021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.703096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.703171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.703245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.703319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.703401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.703482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.703564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.703646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.703734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.703812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.703890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.703967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.704047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.704127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.704205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.704284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.704362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.704450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.704539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.704627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.704716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.704797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.704874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.704951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.705028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.705107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.705185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.705264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.705340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.705420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.705499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.705581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.705660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.705751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.705829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.705906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.705986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.706064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.706143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.706220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.706298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.706375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.706456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.706539] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.706651] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.706655] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54959, lrc_seqno=54959, guc_id=0, flags=0x73 in no process [-1]
<7> [178.706658] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.707586] ------------[ cut here ]------------
<4> [178.707588] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.707589] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.707664] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.707733] efi_pstore nfnetlink autofs4
<4> [178.707738] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.707741] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.707742] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.707743] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.707748] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.707817] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.707819] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.707821] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.707823] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.707824] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.707825] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.707826] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.707828] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.707829] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.707831] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.707832] PKRU: 55555554
<4> [178.707833] Call Trace:
<4> [178.707834] <TASK>
<4> [178.707838] ? lock_sync+0x100/0x100
<4> [178.707843] ? lock_release+0xd0/0x2b0
<4> [178.707848] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.707854] process_one_work+0x239/0x760
<4> [178.707860] worker_thread+0x200/0x3f0
<4> [178.707863] ? __pfx_worker_thread+0x10/0x10
<4> [178.707865] kthread+0x10d/0x150
<4> [178.707868] ? __pfx_kthread+0x10/0x10
<4> [178.707872] ret_from_fork+0x3d4/0x480
<4> [178.707874] ? __pfx_kthread+0x10/0x10
<4> [178.707877] ret_from_fork_asm+0x1a/0x30
<4> [178.707884] </TASK>
<4> [178.707885] irq event stamp: 189043
<4> [178.707887] hardirqs last enabled at (189049): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.707889] hardirqs last disabled at (189054): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.707892] softirqs last enabled at (187938): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.707894] softirqs last disabled at (187929): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.707897] ---[ end trace 0000000000000000 ]---
<7> [178.707910] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.708012] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.708109] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.708181] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.708187] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.708596] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.708796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.708883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.708975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.709065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.709152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.709239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.709326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.709415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.709500] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.709595] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.709716] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.710724] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.720681] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.720929] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.721969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.722048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.722125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.722203] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.722282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.722362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.722439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.722515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.722591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.722665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.722754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.722830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.722904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.722979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.723054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.723128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.723202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.723276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.723351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.723433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.723515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.723598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.723686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.723767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.723846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.723923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.724000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.724080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.724160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.724238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.724321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.724404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.724492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.724578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.724664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.724760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.724843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.724924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.725005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.725085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.725167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.725246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.725324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.725400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.725481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.725559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.725641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.725729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.725811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.725888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.725965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.726044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.726123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.726201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.726279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.726357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.726435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.726515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.726599] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.726715] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.726719] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=54959, lrc_seqno=54959, guc_id=0, flags=0x73 in no process [-1]
<7> [178.726722] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.726780] ------------[ cut here ]------------
<4> [178.726781] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.726783] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.726854] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.726918] efi_pstore nfnetlink autofs4
<4> [178.726922] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.726925] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.726926] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.726927] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.726932] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.727002] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.727003] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.727005] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.727007] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.727008] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.727009] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.727010] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.727012] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.727013] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.727014] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.727016] PKRU: 55555554
<4> [178.727017] Call Trace:
<4> [178.727018] <TASK>
<4> [178.727022] ? lock_sync+0x100/0x100
<4> [178.727026] ? lock_release+0xd0/0x2b0
<4> [178.727031] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.727037] process_one_work+0x239/0x760
<4> [178.727043] worker_thread+0x200/0x3f0
<4> [178.727046] ? __pfx_worker_thread+0x10/0x10
<4> [178.727048] kthread+0x10d/0x150
<4> [178.727051] ? __pfx_kthread+0x10/0x10
<4> [178.727054] ret_from_fork+0x3d4/0x480
<4> [178.727056] ? __pfx_kthread+0x10/0x10
<4> [178.727060] ret_from_fork_asm+0x1a/0x30
<4> [178.727067] </TASK>
<4> [178.727068] irq event stamp: 192131
<4> [178.727069] hardirqs last enabled at (192137): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.727072] hardirqs last disabled at (192142): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.727074] softirqs last enabled at (191128): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.727077] softirqs last disabled at (191119): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.727079] ---[ end trace 0000000000000000 ]---
<5> [178.728602] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55014, lrc_seqno=55014, guc_id=0, flags=0x73 in no process [-1]
<7> [178.728608] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.728725] ------------[ cut here ]------------
<4> [178.728727] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [178.728681] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [178.728729] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:45/3063
<7> [178.728786] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [178.728849] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.728941] efi_pstore nfnetlink autofs4
<4> [178.728947] CPU: 12 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.728951] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.728953] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.728955] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.728963] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.729077] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.729080] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.729083] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.729085] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.729087] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.729089] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.729091] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.729093] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [178.729095] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.729097] CR2: 0000768ff00973c8 CR3: 000000012cae9004 CR4: 0000000000f72ef0
<4> [178.729099] PKRU: 55555554
<4> [178.729101] Call Trace:
<4> [178.729103] <TASK>
<4> [178.729109] ? lock_sync+0x100/0x100
<4> [178.729116] ? lock_release+0xd0/0x2b0
<4> [178.729125] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.729134] process_one_work+0x239/0x760
<4> [178.729144] worker_thread+0x200/0x3f0
<4> [178.729148] ? __pfx_worker_thread+0x10/0x10
<4> [178.729152] kthread+0x10d/0x150
<4> [178.729156] ? __pfx_kthread+0x10/0x10
<4> [178.729161] ret_from_fork+0x3d4/0x480
<4> [178.729164] ? __pfx_kthread+0x10/0x10
<4> [178.729169] ret_from_fork_asm+0x1a/0x30
<4> [178.729181] </TASK>
<4> [178.729182] irq event stamp: 604585
<4> [178.729184] hardirqs last enabled at (604591): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.729188] hardirqs last disabled at (604596): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.729191] softirqs last enabled at (598980): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70
<4> [178.729194] softirqs last disabled at (598978): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120
<4> [178.729198] ---[ end trace 0000000000000000 ]---
<6> [178.729200] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.729315] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.729324] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.729543] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.729796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.729931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.730070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.730204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.730337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.730468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.730601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.730742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.730874] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.731016] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.731181] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.732555] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.742682] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.742993] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.744169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.744290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.744409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.744530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.744650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.744797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.744918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.745039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.745159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.745279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.745400] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.745520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.745640] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.745766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.745887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.746008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.746128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.746248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.746368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.746493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.746618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.746749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.746876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.746999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.747122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.747246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.747368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.747495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.747622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.747752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.747879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.748005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.748137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.748268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.748399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.748527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.748655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.748784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.748909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.749033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.749160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.749286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.749412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.749536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.749664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.749793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.749921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.750045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.750170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.750293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.750417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.750544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.750668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.750797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.750921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.751046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.751169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.751292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.751420] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.751580] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.751586] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55014, lrc_seqno=55014, guc_id=0, flags=0x73 in no process [-1]
<7> [178.751589] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.751696] ------------[ cut here ]------------
<4> [178.751697] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.751699] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:45/3063
<4> [178.751814] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.751903] efi_pstore nfnetlink autofs4
<4> [178.751909] CPU: 12 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.751913] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.751915] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.751917] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.751924] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.752035] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.752038] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.752041] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.752043] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.752045] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.752046] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.752048] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.752050] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000
<4> [178.752052] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.752054] CR2: 0000768ff00973c8 CR3: 000000012cae9004 CR4: 0000000000f72ef0
<4> [178.752056] PKRU: 55555554
<4> [178.752058] Call Trace:
<4> [178.752060] <TASK>
<4> [178.752066] ? lock_sync+0x100/0x100
<4> [178.752073] ? lock_release+0xd0/0x2b0
<4> [178.752081] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.752090] process_one_work+0x239/0x760
<4> [178.752100] worker_thread+0x200/0x3f0
<4> [178.752105] ? __pfx_worker_thread+0x10/0x10
<4> [178.752108] kthread+0x10d/0x150
<4> [178.752112] ? __pfx_kthread+0x10/0x10
<4> [178.752117] ret_from_fork+0x3d4/0x480
<4> [178.752120] ? __pfx_kthread+0x10/0x10
<4> [178.752125] ret_from_fork_asm+0x1a/0x30
<4> [178.752137] </TASK>
<4> [178.752138] irq event stamp: 607743
<4> [178.752140] hardirqs last enabled at (607749): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.752144] hardirqs last disabled at (607754): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.752147] softirqs last enabled at (606628): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.752150] softirqs last disabled at (606623): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.752153] ---[ end trace 0000000000000000 ]---
<6> [178.752166] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.752244] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.752251] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.753885] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.754052] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.754208] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.754414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.754505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.754602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.754695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.754784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.754870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.754956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.755044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.755126] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.755222] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.755342] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.756362] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.766676] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.766932] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.767940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.768021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.768099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.768180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.768258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.768336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.768412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.768488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.768562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.768636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.768750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.768825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.768899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.768974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.769047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.769123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.769199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.769274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.769348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.769431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.769511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.769590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.769677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.769758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.769837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.769916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.769994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.770073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.770153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.770232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.770312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.770391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.770475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.770559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.770643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.770732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.770813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.770891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.770968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.771045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.771125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.771204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.771282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.771358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.771439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.771522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.771604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.771685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.771765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.771841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.771918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.772003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.772085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.772163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.772241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.772319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.772397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.772475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.772556] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.772672] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.772677] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55014, lrc_seqno=55014, guc_id=0, flags=0x73 in no process [-1]
<7> [178.772680] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.772739] ------------[ cut here ]------------
<4> [178.772741] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.772742] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:45/3063
<4> [178.772814] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.772880] efi_pstore nfnetlink autofs4
<4> [178.772884] CPU: 6 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.772887] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.772888] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.772890] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.772895] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.772966] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.772967] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.772970] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.772971] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.772973] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.772974] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.772975] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.772976] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.772978] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.772979] CR2: 000077a5ba55f000 CR3: 0000000115508005 CR4: 0000000000f72ef0
<4> [178.772981] PKRU: 55555554
<4> [178.772982] Call Trace:
<4> [178.772983] <TASK>
<4> [178.772987] ? lock_sync+0x100/0x100
<4> [178.772992] ? lock_release+0xd0/0x2b0
<4> [178.772998] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.773003] process_one_work+0x239/0x760
<4> [178.773010] worker_thread+0x200/0x3f0
<4> [178.773013] ? __pfx_worker_thread+0x10/0x10
<4> [178.773015] kthread+0x10d/0x150
<4> [178.773018] ? __pfx_kthread+0x10/0x10
<4> [178.773022] ret_from_fork+0x3d4/0x480
<4> [178.773024] ? __pfx_kthread+0x10/0x10
<4> [178.773027] ret_from_fork_asm+0x1a/0x30
<4> [178.773035] </TASK>
<4> [178.773036] irq event stamp: 611067
<4> [178.773037] hardirqs last enabled at (611073): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.773040] hardirqs last disabled at (611078): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.773042] softirqs last enabled at (609964): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.773045] softirqs last disabled at (609957): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.773047] ---[ end trace 0000000000000000 ]---
<5> [178.773984] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55069, lrc_seqno=55069, guc_id=0, flags=0x73 in no process [-1]
<7> [178.773992] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.774127] ------------[ cut here ]------------
<4> [178.774129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.774132] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.774226] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.774294] efi_pstore nfnetlink autofs4
<4> [178.774299] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.774302] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.774303] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.774304] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.774310] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.774386] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.774388] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.774390] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.774392] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.774393] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.774394] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.774395] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.774397] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.774398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.774400] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [178.774401] PKRU: 55555554
<4> [178.774402] Call Trace:
<4> [178.774404] <TASK>
<4> [178.774408] ? lock_sync+0x100/0x100
<4> [178.774412] ? lock_release+0xd0/0x2b0
<4> [178.774418] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.774423] process_one_work+0x239/0x760
<4> [178.774430] worker_thread+0x200/0x3f0
<4> [178.774433] ? __pfx_worker_thread+0x10/0x10
<4> [178.774435] kthread+0x10d/0x150
<4> [178.774438] ? __pfx_kthread+0x10/0x10
<4> [178.774442] ret_from_fork+0x3d4/0x480
<4> [178.774444] ? __pfx_kthread+0x10/0x10
<4> [178.774447] ret_from_fork_asm+0x1a/0x30
<4> [178.774455] </TASK>
<4> [178.774456] irq event stamp: 2497641
<4> [178.774457] hardirqs last enabled at (2497647): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.774460] hardirqs last disabled at (2497652): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.774462] softirqs last enabled at (2496012): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.774465] softirqs last disabled at (2496005): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.774467] ---[ end trace 0000000000000000 ]---
<6> [178.774469] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.774545] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.774552] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.774550] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.774657] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.774821] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.775045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.775146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.775245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.775338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.775429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.775521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.775611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.775713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.775800] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.775901] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.776023] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.777039] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.787677] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.787945] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.788991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.789080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.789164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.789247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.789328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.789408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.789487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.789565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.789644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.789733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.789812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.789891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.789971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.790049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.790128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.790209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.790291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.790371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.790450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.790537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.790623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.790716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.790802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.790886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.790969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.791056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.791141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.791225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.791312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.791396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.791480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.791565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.791656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.791757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.791846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.791932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.792021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.792107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.792193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.792276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.792361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.792445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.792528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.792611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.792701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.792784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.792866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.792948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.793030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.793110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.793190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.793274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.793356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.793438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.793518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.793600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.793683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.793763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.793847] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.793963] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.793968] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55069, lrc_seqno=55069, guc_id=0, flags=0x73 in no process [-1]
<7> [178.793971] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.794033] ------------[ cut here ]------------
<4> [178.794034] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.794035] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.794113] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.794178] efi_pstore nfnetlink autofs4
<4> [178.794183] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.794186] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.794187] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.794188] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.794194] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.794267] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.794269] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.794271] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.794273] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.794274] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.794276] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.794277] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.794278] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.794280] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.794281] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [178.794283] PKRU: 55555554
<4> [178.794284] Call Trace:
<4> [178.794285] <TASK>
<4> [178.794289] ? lock_sync+0x100/0x100
<4> [178.794293] ? lock_release+0xd0/0x2b0
<4> [178.794299] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.794305] process_one_work+0x239/0x760
<4> [178.794311] worker_thread+0x200/0x3f0
<4> [178.794314] ? __pfx_worker_thread+0x10/0x10
<4> [178.794316] kthread+0x10d/0x150
<4> [178.794319] ? __pfx_kthread+0x10/0x10
<4> [178.794323] ret_from_fork+0x3d4/0x480
<4> [178.794325] ? __pfx_kthread+0x10/0x10
<4> [178.794328] ret_from_fork_asm+0x1a/0x30
<4> [178.794335] </TASK>
<4> [178.794336] irq event stamp: 2500743
<4> [178.794337] hardirqs last enabled at (2500749): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.794340] hardirqs last disabled at (2500754): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.794343] softirqs last enabled at (2499634): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.794345] softirqs last disabled at (2499627): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.794347] ---[ end trace 0000000000000000 ]---
<6> [178.794349] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.794424] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.794430] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.795675] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.795787] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.795897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.796103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.796194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.796288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.796377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.796464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.796551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.796639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.796756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.796838] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.796935] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.797052] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.798059] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.808672] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.808921] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.809932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.810013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.810092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.810170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.810247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.810323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.810399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.810474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.810549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.810624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.810718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.810796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.810874] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.810949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.811024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.811098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.811172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.811247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.811322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.811405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.811486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.811566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.811650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.811741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.811819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.811898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.811976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.812056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.812136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.812215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.812294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.812374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.812463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.812551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.812637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.812728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.812809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.812887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.812966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.813044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.813124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.813203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.813282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.813360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.813441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.813519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.813597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.813676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.813756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.813831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.813909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.813988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.814065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.814143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.814220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.814297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.814374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.814450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.814529] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.814641] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.814645] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55069, lrc_seqno=55069, guc_id=0, flags=0x73 in no process [-1]
<7> [178.814647] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.814725] ------------[ cut here ]------------
<4> [178.814726] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.814728] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.814801] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.814866] efi_pstore nfnetlink autofs4
<4> [178.814870] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.814873] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.814874] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.814876] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.814881] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.814951] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.814953] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.814955] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.814957] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.814958] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.814959] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.814960] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.814962] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.814963] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.814965] CR2: 000060dda491f42c CR3: 0000000114e1a005 CR4: 0000000000f72ef0
<4> [178.814966] PKRU: 55555554
<4> [178.814968] Call Trace:
<4> [178.814969] <TASK>
<4> [178.814973] ? lock_sync+0x100/0x100
<4> [178.814977] ? lock_release+0xd0/0x2b0
<4> [178.814983] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.814988] process_one_work+0x239/0x760
<4> [178.814994] worker_thread+0x200/0x3f0
<4> [178.814997] ? __pfx_worker_thread+0x10/0x10
<4> [178.814999] kthread+0x10d/0x150
<4> [178.815002] ? __pfx_kthread+0x10/0x10
<4> [178.815006] ret_from_fork+0x3d4/0x480
<4> [178.815008] ? __pfx_kthread+0x10/0x10
<4> [178.815011] ret_from_fork_asm+0x1a/0x30
<4> [178.815019] </TASK>
<4> [178.815020] irq event stamp: 201457
<4> [178.815021] hardirqs last enabled at (201463): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.815024] hardirqs last disabled at (201468): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.815026] softirqs last enabled at (200454): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.815028] softirqs last disabled at (200449): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.815031] ---[ end trace 0000000000000000 ]---
<5> [178.815626] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55124, lrc_seqno=55124, guc_id=0, flags=0x73 in no process [-1]
<7> [178.815628] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.816536] ------------[ cut here ]------------
<4> [178.816538] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.816539] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.816611] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.816678] efi_pstore nfnetlink autofs4
<4> [178.816683] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.816686] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.816688] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.816689] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.816693] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.816762] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.816764] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.816766] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.816767] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.816768] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.816769] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.816771] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.816772] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.816773] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.816775] CR2: 000060dda491f42c CR3: 0000000114e1a005 CR4: 0000000000f72ef0
<4> [178.816776] PKRU: 55555554
<4> [178.816777] Call Trace:
<4> [178.816778] <TASK>
<4> [178.816782] ? lock_sync+0x100/0x100
<4> [178.816786] ? lock_release+0xd0/0x2b0
<4> [178.816792] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.816797] process_one_work+0x239/0x760
<4> [178.816803] worker_thread+0x200/0x3f0
<4> [178.816806] ? __pfx_worker_thread+0x10/0x10
<4> [178.816808] kthread+0x10d/0x150
<4> [178.816811] ? __pfx_kthread+0x10/0x10
<4> [178.816814] ret_from_fork+0x3d4/0x480
<4> [178.816816] ? __pfx_kthread+0x10/0x10
<4> [178.816819] ret_from_fork_asm+0x1a/0x30
<4> [178.816827] </TASK>
<4> [178.816828] irq event stamp: 204229
<4> [178.816829] hardirqs last enabled at (204235): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.816831] hardirqs last disabled at (204240): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.816833] softirqs last enabled at (200454): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.816836] softirqs last disabled at (200449): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.816838] ---[ end trace 0000000000000000 ]---
<7> [178.816851] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.816952] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.817051] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.817120] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.817126] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.817227] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.817433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.817525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.817620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.817721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.817809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.817895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.817982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.818074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.818159] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.818255] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.818370] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.819378] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.829668] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.829920] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.830898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.830981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.831060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.831137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.831214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.831290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.831365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.831441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.831519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.831595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.831676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.831751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.831827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.831902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.831977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.832051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.832127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.832201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.832274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.832356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.832441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.832523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.832604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.832686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.832764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.832842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.832919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.832998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.833078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.833161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.833242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.833322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.833407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.833492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.833576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.833657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.833752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.833834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.833913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.833992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.834072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.834150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.834227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.834304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.834384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.834460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.834538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.834614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.834703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.834783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.834861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.834942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.835021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.835099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.835175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.835251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.835326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.835400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.835478] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.835590] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.835594] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55124, lrc_seqno=55124, guc_id=0, flags=0x73 in no process [-1]
<7> [178.835596] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.835655] ------------[ cut here ]------------
<4> [178.835656] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.835658] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:45/3063
<4> [178.835746] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.835812] efi_pstore nfnetlink autofs4
<4> [178.835817] CPU: 6 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.835819] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.835821] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.835822] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.835827] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.835896] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.835898] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.835901] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.835902] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.835903] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.835905] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.835906] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.835907] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.835909] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.835910] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.835912] PKRU: 55555554
<4> [178.835913] Call Trace:
<4> [178.835914] <TASK>
<4> [178.835918] ? lock_sync+0x100/0x100
<4> [178.835922] ? lock_release+0xd0/0x2b0
<4> [178.835928] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.835933] process_one_work+0x239/0x760
<4> [178.835940] worker_thread+0x200/0x3f0
<4> [178.835942] ? __pfx_worker_thread+0x10/0x10
<4> [178.835945] kthread+0x10d/0x150
<4> [178.835948] ? __pfx_kthread+0x10/0x10
<4> [178.835951] ret_from_fork+0x3d4/0x480
<4> [178.835954] ? __pfx_kthread+0x10/0x10
<4> [178.835957] ret_from_fork_asm+0x1a/0x30
<4> [178.835964] </TASK>
<4> [178.835965] irq event stamp: 616821
<4> [178.835966] hardirqs last enabled at (616827): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.835969] hardirqs last disabled at (616832): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.835971] softirqs last enabled at (616078): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.835974] softirqs last disabled at (616061): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.835976] ---[ end trace 0000000000000000 ]---
<6> [178.835978] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.836048] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.836054] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.837290] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.837395] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.837502] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.837710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.837802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.837895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.837984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.838071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.838158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.838245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.838332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.838416] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.838512] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.838630] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.839642] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.850377] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.850625] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.851649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.851742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.851819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.851896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.851974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.852050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.852125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.852199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.852273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.852346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.852419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.852492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.852568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.852647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.852738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.852815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.852890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.852966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.853040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.853121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.853203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.853282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.853362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.853441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.853519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.853601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.853687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.853769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.853850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.853930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.854013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.854095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.854181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.854266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.854352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.854433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.854512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.854591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.854677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.854756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.854838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.854917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.854994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.855072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.855153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.855229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.855305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.855384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.855465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.855542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.855619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.855706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.855785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.855862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.855941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.856023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.856102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.856180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.856259] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.856370] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.856374] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55124, lrc_seqno=55124, guc_id=0, flags=0x73 in no process [-1]
<7> [178.856377] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.856434] ------------[ cut here ]------------
<4> [178.856436] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.856437] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.856508] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.856573] efi_pstore nfnetlink autofs4
<4> [178.856577] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.856580] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.856581] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.856583] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.856588] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.856660] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.856662] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.856664] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.856666] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.856667] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.856669] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.856670] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.856672] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.856673] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.856675] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [178.856676] PKRU: 55555554
<4> [178.856678] Call Trace:
<4> [178.856679] <TASK>
<4> [178.856683] ? lock_sync+0x100/0x100
<4> [178.856687] ? lock_release+0xd0/0x2b0
<4> [178.856693] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.856699] process_one_work+0x239/0x760
<4> [178.856705] worker_thread+0x200/0x3f0
<4> [178.856708] ? __pfx_worker_thread+0x10/0x10
<4> [178.856710] kthread+0x10d/0x150
<4> [178.856713] ? __pfx_kthread+0x10/0x10
<4> [178.856717] ret_from_fork+0x3d4/0x480
<4> [178.856719] ? __pfx_kthread+0x10/0x10
<4> [178.856722] ret_from_fork_asm+0x1a/0x30
<4> [178.856730] </TASK>
<4> [178.856731] irq event stamp: 2504261
<4> [178.856732] hardirqs last enabled at (2504267): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.856735] hardirqs last disabled at (2504272): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.856737] softirqs last enabled at (2503182): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.856740] softirqs last disabled at (2503175): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.856742] ---[ end trace 0000000000000000 ]---
<5> [178.858248] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55179, lrc_seqno=55179, guc_id=0, flags=0x73 in no process [-1]
<7> [178.858251] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.858314] ------------[ cut here ]------------
<4> [178.858315] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.858316] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.858387] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.858450] efi_pstore nfnetlink autofs4
<4> [178.858454] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.858456] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.858457] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.858459] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.858463] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.858534] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.858536] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.858538] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.858540] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.858541] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.858542] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.858543] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.858544] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.858546] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.858547] CR2: 000077a5bc1e62f4 CR3: 000000000344c005 CR4: 0000000000f72ef0
<4> [178.858549] PKRU: 55555554
<4> [178.858550] Call Trace:
<4> [178.858551] <TASK>
<4> [178.858555] ? lock_sync+0x100/0x100
<4> [178.858559] ? lock_release+0xd0/0x2b0
<4> [178.858564] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.858570] process_one_work+0x239/0x760
<4> [178.858576] worker_thread+0x200/0x3f0
<4> [178.858578] ? __pfx_worker_thread+0x10/0x10
<4> [178.858581] kthread+0x10d/0x150
<4> [178.858584] ? __pfx_kthread+0x10/0x10
<4> [178.858587] ret_from_fork+0x3d4/0x480
<4> [178.858589] ? __pfx_kthread+0x10/0x10
<4> [178.858592] ret_from_fork_asm+0x1a/0x30
<4> [178.858600] </TASK>
<4> [178.858601] irq event stamp: 2507037
<4> [178.858602] hardirqs last enabled at (2507043): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.858604] hardirqs last disabled at (2507048): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.858607] softirqs last enabled at (2503182): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.858609] softirqs last disabled at (2503175): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.858611] ---[ end trace 0000000000000000 ]---
<6> [178.858613] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [178.858709] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.858813] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.858912] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.858919] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.858997] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.859205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.859297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.859394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.859484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.859576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.859669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.859757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.859846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.859929] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.860024] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.860141] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.861149] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.871663] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.871913] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.872904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.872984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.873063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.873141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.873218] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.873293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.873368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.873443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.873518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.873592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.873670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.873745] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.873819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.873894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.873971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.874048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.874125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.874199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.874273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.874355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.874436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.874514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.874593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.874673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.874751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.874830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.874912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.874995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.875077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.875156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.875237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.875318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.875403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.875488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.875573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.875663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.875748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.875828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.875908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.875986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.876067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.876146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.876223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.876299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.876381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.876458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.876535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.876611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.876731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.876813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.876894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.876976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.877055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.877133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.877210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.877288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.877365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.877441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.877525] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.877639] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.877643] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55179, lrc_seqno=55179, guc_id=0, flags=0x73 in no process [-1]
<7> [178.877646] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.878572] ------------[ cut here ]------------
<4> [178.878573] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.878575] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.878648] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.878717] efi_pstore nfnetlink autofs4
<4> [178.878721] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.878724] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.878725] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.878727] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.878732] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.878800] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.878802] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.878804] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.878806] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.878807] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.878808] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.878809] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.878811] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.878812] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.878814] CR2: 000060dda491f42c CR3: 000000012cae9001 CR4: 0000000000f72ef0
<4> [178.878815] PKRU: 55555554
<4> [178.878816] Call Trace:
<4> [178.878818] <TASK>
<4> [178.878821] ? lock_sync+0x100/0x100
<4> [178.878826] ? lock_release+0xd0/0x2b0
<4> [178.878831] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.878837] process_one_work+0x239/0x760
<4> [178.878843] worker_thread+0x200/0x3f0
<4> [178.878846] ? __pfx_worker_thread+0x10/0x10
<4> [178.878848] kthread+0x10d/0x150
<4> [178.878851] ? __pfx_kthread+0x10/0x10
<4> [178.878855] ret_from_fork+0x3d4/0x480
<4> [178.878857] ? __pfx_kthread+0x10/0x10
<4> [178.878860] ret_from_fork_asm+0x1a/0x30
<4> [178.878867] </TASK>
<4> [178.878869] irq event stamp: 209811
<4> [178.878870] hardirqs last enabled at (209817): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.878873] hardirqs last disabled at (209822): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.878875] softirqs last enabled at (209044): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.878877] softirqs last disabled at (209037): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.878880] ---[ end trace 0000000000000000 ]---
<6> [178.878881] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.878951] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.878957] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.878971] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.879075] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.879363] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.879568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.879663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.879758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.879847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.879936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.880021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.880107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.880193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.880274] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.880369] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.880484] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.881504] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.891660] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.891909] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.892925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.893005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.893081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.893157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.893233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.893308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.893383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.893459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.893534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.893609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.893692] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.893766] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.893841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.893915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.893989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.894063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.894136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.894209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.894282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.894364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.894446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.894529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.894611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.894698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.894776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.894853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.894929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.895008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.895089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.895168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.895250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.895333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.895420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.895506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.895590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.895675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.895760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.895838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.895917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.895995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.896075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.896157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.896238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.896316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.896397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.896473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.896550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.896625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.896713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.896793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.896872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.896952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.897030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.897107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.897184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.897261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.897336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.897410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.897489] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.897600] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.897604] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55179, lrc_seqno=55179, guc_id=0, flags=0x73 in no process [-1]
<7> [178.897606] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.897679] ------------[ cut here ]------------
<4> [178.897680] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.897681] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:45/3063
<4> [178.897752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.897816] efi_pstore nfnetlink autofs4
<4> [178.897820] CPU: 6 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.897823] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.897824] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.897826] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.897831] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.897901] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.897903] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.897905] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.897906] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.897908] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.897909] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.897910] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.897912] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.897913] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.897915] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.897916] PKRU: 55555554
<4> [178.897917] Call Trace:
<4> [178.897918] <TASK>
<4> [178.897922] ? lock_sync+0x100/0x100
<4> [178.897927] ? lock_release+0xd0/0x2b0
<4> [178.897932] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.897938] process_one_work+0x239/0x760
<4> [178.897944] worker_thread+0x200/0x3f0
<4> [178.897946] ? __pfx_worker_thread+0x10/0x10
<4> [178.897949] kthread+0x10d/0x150
<4> [178.897952] ? __pfx_kthread+0x10/0x10
<4> [178.897955] ret_from_fork+0x3d4/0x480
<4> [178.897957] ? __pfx_kthread+0x10/0x10
<4> [178.897961] ret_from_fork_asm+0x1a/0x30
<4> [178.897968] </TASK>
<4> [178.897969] irq event stamp: 620259
<4> [178.897970] hardirqs last enabled at (620265): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.897973] hardirqs last disabled at (620270): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.897975] softirqs last enabled at (619498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.897978] softirqs last disabled at (619465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.897980] ---[ end trace 0000000000000000 ]---
<5> [178.899489] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55234, lrc_seqno=55234, guc_id=0, flags=0x73 in no process [-1]
<7> [178.899492] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.899554] ------------[ cut here ]------------
<4> [178.899555] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.899556] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:45/3063
<4> [178.899626] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.899694] efi_pstore nfnetlink autofs4
<4> [178.899698] CPU: 6 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.899701] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.899702] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.899703] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.899708] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.899776] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.899778] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.899780] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.899781] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.899783] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.899784] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.899785] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.899786] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.899788] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.899789] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.899791] PKRU: 55555554
<4> [178.899792] Call Trace:
<4> [178.899793] <TASK>
<4> [178.899797] ? lock_sync+0x100/0x100
<4> [178.899801] ? lock_release+0xd0/0x2b0
<4> [178.899806] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.899812] process_one_work+0x239/0x760
<4> [178.899818] worker_thread+0x200/0x3f0
<4> [178.899821] ? __pfx_worker_thread+0x10/0x10
<4> [178.899823] kthread+0x10d/0x150
<4> [178.899826] ? __pfx_kthread+0x10/0x10
<4> [178.899829] ret_from_fork+0x3d4/0x480
<4> [178.899831] ? __pfx_kthread+0x10/0x10
<4> [178.899835] ret_from_fork_asm+0x1a/0x30
<4> [178.899842] </TASK>
<4> [178.899843] irq event stamp: 622971
<4> [178.899844] hardirqs last enabled at (622977): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.899846] hardirqs last disabled at (622982): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.899849] softirqs last enabled at (619498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.899851] softirqs last disabled at (619465): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.899853] ---[ end trace 0000000000000000 ]---
<7> [178.899867] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.899970] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.900069] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.900138] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.900143] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.900190] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.900392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.900480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.900573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.900667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.900758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.900844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.900930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.901018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.901100] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.901197] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.901315] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.902323] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.912658] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.912909] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.913928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.914007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.914084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.914163] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.914243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.914322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.914399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.914476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.914551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.914626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.914711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.914786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.914861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.914935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.915009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.915085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.915162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.915238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.915312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.915394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.915476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.915556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.915639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.915731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.915810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.915888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.915966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.916045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.916126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.916205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.916285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.916365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.916450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.916534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.916622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.916714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.916796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.916874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.916952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.917027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.917106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.917183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.917261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.917341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.917425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.917503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.917581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.917659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.917738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.917813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.917888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.917967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.918043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.918119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.918198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.918280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.918359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.918436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.918516] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.918627] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.918630] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55234, lrc_seqno=55234, guc_id=0, flags=0x73 in no process [-1]
<7> [178.918633] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.918711] ------------[ cut here ]------------
<4> [178.918712] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.918714] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:45/3063
<4> [178.918785] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.918849] efi_pstore nfnetlink autofs4
<4> [178.918854] CPU: 6 UID: 0 PID: 3063 Comm: kworker/u64:45 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.918856] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.918857] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.918859] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.918864] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.918933] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.918935] RSP: 0018:ffffc90002b53ca0 EFLAGS: 00010246
<4> [178.918937] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.918939] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.918940] RBP: ffffc90002b53db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.918941] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.918942] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.918944] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000
<4> [178.918945] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.918947] CR2: 000077a5ba55f000 CR3: 000000012cae9003 CR4: 0000000000f72ef0
<4> [178.918948] PKRU: 55555554
<4> [178.918949] Call Trace:
<4> [178.918950] <TASK>
<4> [178.918954] ? lock_sync+0x100/0x100
<4> [178.918959] ? lock_release+0xd0/0x2b0
<4> [178.918964] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.918970] process_one_work+0x239/0x760
<4> [178.918976] worker_thread+0x200/0x3f0
<4> [178.918978] ? __pfx_worker_thread+0x10/0x10
<4> [178.918981] kthread+0x10d/0x150
<4> [178.918983] ? __pfx_kthread+0x10/0x10
<4> [178.918987] ret_from_fork+0x3d4/0x480
<4> [178.918989] ? __pfx_kthread+0x10/0x10
<4> [178.918993] ret_from_fork_asm+0x1a/0x30
<4> [178.919000] </TASK>
<4> [178.919001] irq event stamp: 626057
<4> [178.919002] hardirqs last enabled at (626063): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.919005] hardirqs last disabled at (626068): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.919007] softirqs last enabled at (625296): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.919009] softirqs last disabled at (625285): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.919012] ---[ end trace 0000000000000000 ]---
<6> [178.919013] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.919083] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.919088] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.920321] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.920427] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.920532] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.920742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.920833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.920927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.921016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.921103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.921187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.921274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.921361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.921444] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.921539] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.921660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.922673] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.933394] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.933642] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.934693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.934773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.934850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.934928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.935005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.935081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.935156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.935229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.935302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.935375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.935448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.935521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.935594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.935672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.935746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.935819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.935892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.935965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.936039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.936121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.936203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.936284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.936369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.936449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.936529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.936608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.936694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.936774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.936855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.936935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.937014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.937094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.937180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.937264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.937348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.937429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.937509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.937587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.937667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.937747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.937832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.937914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.937995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.938077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.938161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.938240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.938320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.938396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.938474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.938552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.938628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.938716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.938794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.938872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.938949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.939027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.939103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.939179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.939260] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.939371] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.939375] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55234, lrc_seqno=55234, guc_id=0, flags=0x73 in no process [-1]
<7> [178.939378] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.939436] ------------[ cut here ]------------
<4> [178.939438] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.939439] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.939511] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.939575] efi_pstore nfnetlink autofs4
<4> [178.939579] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.939582] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.939583] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.939584] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.939589] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.939662] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.939664] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.939667] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.939668] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.939670] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.939671] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.939672] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.939674] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.939675] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.939677] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.939679] PKRU: 55555554
<4> [178.939680] Call Trace:
<4> [178.939681] <TASK>
<4> [178.939685] ? lock_sync+0x100/0x100
<4> [178.939689] ? lock_release+0xd0/0x2b0
<4> [178.939695] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.939701] process_one_work+0x239/0x760
<4> [178.939707] worker_thread+0x200/0x3f0
<4> [178.939710] ? __pfx_worker_thread+0x10/0x10
<4> [178.939712] kthread+0x10d/0x150
<4> [178.939715] ? __pfx_kthread+0x10/0x10
<4> [178.939719] ret_from_fork+0x3d4/0x480
<4> [178.939721] ? __pfx_kthread+0x10/0x10
<4> [178.939724] ret_from_fork_asm+0x1a/0x30
<4> [178.939732] </TASK>
<4> [178.939733] irq event stamp: 213255
<4> [178.939734] hardirqs last enabled at (213261): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.939737] hardirqs last disabled at (213266): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.939739] softirqs last enabled at (212176): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.939742] softirqs last disabled at (212169): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.939744] ---[ end trace 0000000000000000 ]---
<5> [178.940336] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55289, lrc_seqno=55289, guc_id=0, flags=0x73 in no process [-1]
<7> [178.940339] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.940398] ------------[ cut here ]------------
<4> [178.940399] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.940400] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.940471] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.940534] efi_pstore nfnetlink autofs4
<4> [178.940538] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.940540] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.940541] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.940542] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.940547] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.940616] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.940618] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.940620] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.940621] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.940622] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.940624] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.940625] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.940626] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.940628] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.940629] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.940630] PKRU: 55555554
<4> [178.940631] Call Trace:
<4> [178.940632] <TASK>
<4> [178.940636] ? lock_sync+0x100/0x100
<4> [178.940640] ? lock_release+0xd0/0x2b0
<4> [178.940645] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.940654] process_one_work+0x239/0x760
<4> [178.940660] worker_thread+0x200/0x3f0
<4> [178.940663] ? __pfx_worker_thread+0x10/0x10
<4> [178.940665] kthread+0x10d/0x150
<4> [178.940668] ? __pfx_kthread+0x10/0x10
<4> [178.940672] ret_from_fork+0x3d4/0x480
<4> [178.940674] ? __pfx_kthread+0x10/0x10
<4> [178.940677] ret_from_fork_asm+0x1a/0x30
<4> [178.940684] </TASK>
<4> [178.940685] irq event stamp: 215979
<4> [178.940686] hardirqs last enabled at (215985): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.940689] hardirqs last disabled at (215990): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.940692] softirqs last enabled at (212176): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.940694] softirqs last disabled at (212169): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.940696] ---[ end trace 0000000000000000 ]---
<6> [178.941518] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.941592] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.941597] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.941657] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.941762] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.941868] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.942071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.942159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.942252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.942340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.942429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.942520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.942609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.942709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.942794] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.942890] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.943007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.944014] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.954654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.954902] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.955921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.956002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.956079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.956157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.956233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.956308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.956384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.956459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.956533] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.956607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.956691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.956769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.956846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.956923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.956999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.957074] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.957149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.957222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.957297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.957378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.957459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.957539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.957624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.957714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.957791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.957868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.957946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.958026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.958106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.958185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.958264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.958343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.958427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.958513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.958600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.958692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.958774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.958853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.958931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.959007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.959087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.959165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.959243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.959319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.959400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.959477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.959554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.959635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.959728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.959806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.959883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.959962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.960039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.960116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.960192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.960270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.960345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.960421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.960499] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.960610] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.960614] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55289, lrc_seqno=55289, guc_id=0, flags=0x73 in no process [-1]
<7> [178.960616] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.961537] ------------[ cut here ]------------
<4> [178.961538] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.961540] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:39/3057
<4> [178.961615] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.961685] efi_pstore nfnetlink autofs4
<4> [178.961689] CPU: 8 UID: 0 PID: 3057 Comm: kworker/u64:39 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.961692] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.961693] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.961695] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.961700] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.961770] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.961771] RSP: 0018:ffffc90002b23ca0 EFLAGS: 00010246
<4> [178.961774] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.961775] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.961776] RBP: ffffc90002b23db0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.961778] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.961779] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.961780] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000
<4> [178.961782] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.961783] CR2: 000060dda491f42c CR3: 000000000344c006 CR4: 0000000000f72ef0
<4> [178.961784] PKRU: 55555554
<4> [178.961786] Call Trace:
<4> [178.961787] <TASK>
<4> [178.961791] ? lock_sync+0x100/0x100
<4> [178.961796] ? lock_release+0xd0/0x2b0
<4> [178.961801] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.961807] process_one_work+0x239/0x760
<4> [178.961813] worker_thread+0x200/0x3f0
<4> [178.961816] ? __pfx_worker_thread+0x10/0x10
<4> [178.961818] kthread+0x10d/0x150
<4> [178.961821] ? __pfx_kthread+0x10/0x10
<4> [178.961825] ret_from_fork+0x3d4/0x480
<4> [178.961827] ? __pfx_kthread+0x10/0x10
<4> [178.961830] ret_from_fork_asm+0x1a/0x30
<4> [178.961838] </TASK>
<4> [178.961839] irq event stamp: 219101
<4> [178.961840] hardirqs last enabled at (219107): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.961843] hardirqs last disabled at (219112): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.961845] softirqs last enabled at (218186): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.961847] softirqs last disabled at (218179): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.961850] ---[ end trace 0000000000000000 ]---
<6> [178.961851] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.961921] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.961927] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.961942] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.962044] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [178.962333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.962540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.962631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.962764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.962855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.962947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.963036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.963123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.963211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.963293] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.963387] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.963504] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.964510] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.974654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.974908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.975895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.975976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.976053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.976131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.976207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.976284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.976359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.976435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.976510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.976586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.976682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.976759] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.976835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.976911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.976986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.977061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.977135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.977210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.977283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.977366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.977448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.977529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.977613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.977706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.977785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.977863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.977940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.978020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.978100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.978178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.978258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.978337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.978421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.978505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.978589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.978677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [178.978756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [178.978833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [178.978911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [178.978987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [178.979066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [178.979143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [178.979221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [178.979302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [178.979386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [178.979465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [178.979543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [178.979621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [178.979709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [178.979786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [178.979862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [178.979941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [178.980019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [178.980097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [178.980172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [178.980249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [178.980324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [178.980400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [178.980478] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [178.980590] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [178.980594] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55289, lrc_seqno=55289, guc_id=0, flags=0x73 in no process [-1]
<7> [178.980597] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.980664] ------------[ cut here ]------------
<4> [178.980665] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.980667] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.980739] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.980806] efi_pstore nfnetlink autofs4
<4> [178.980810] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.980813] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.980814] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.980816] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.980821] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.980892] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.980893] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.980896] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.980897] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.980898] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.980900] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.980901] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.980902] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.980904] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.980905] CR2: 000077a5bc1e62f4 CR3: 0000000114e1a006 CR4: 0000000000f72ef0
<4> [178.980907] PKRU: 55555554
<4> [178.980908] Call Trace:
<4> [178.980909] <TASK>
<4> [178.980913] ? lock_sync+0x100/0x100
<4> [178.980918] ? lock_release+0xd0/0x2b0
<4> [178.980923] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.980929] process_one_work+0x239/0x760
<4> [178.980935] worker_thread+0x200/0x3f0
<4> [178.980938] ? __pfx_worker_thread+0x10/0x10
<4> [178.980941] kthread+0x10d/0x150
<4> [178.980944] ? __pfx_kthread+0x10/0x10
<4> [178.980947] ret_from_fork+0x3d4/0x480
<4> [178.980950] ? __pfx_kthread+0x10/0x10
<4> [178.980953] ret_from_fork_asm+0x1a/0x30
<4> [178.980960] </TASK>
<4> [178.980961] irq event stamp: 2515417
<4> [178.980962] hardirqs last enabled at (2515423): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.980965] hardirqs last disabled at (2515428): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.980968] softirqs last enabled at (2514320): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.980970] softirqs last disabled at (2514313): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.980972] ---[ end trace 0000000000000000 ]---
<5> [178.982536] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55344, lrc_seqno=55344, guc_id=0, flags=0x73 in no process [-1]
<7> [178.982541] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [178.982621] ------------[ cut here ]------------
<4> [178.982622] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [178.982624] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [178.982713] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [178.982778] efi_pstore nfnetlink autofs4
<4> [178.982783] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [178.982786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [178.982787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [178.982788] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [178.982793] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [178.982865] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [178.982866] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [178.982869] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [178.982870] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [178.982871] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [178.982873] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [178.982874] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [178.982875] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [178.982877] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [178.982878] CR2: 000077a5bc1e62f4 CR3: 0000000114e1a006 CR4: 0000000000f72ef0
<4> [178.982879] PKRU: 55555554
<4> [178.982881] Call Trace:
<4> [178.982882] <TASK>
<4> [178.982885] ? lock_sync+0x100/0x100
<4> [178.982890] ? lock_release+0xd0/0x2b0
<4> [178.982895] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [178.982901] process_one_work+0x239/0x760
<4> [178.982907] worker_thread+0x200/0x3f0
<4> [178.982910] ? __pfx_worker_thread+0x10/0x10
<4> [178.982912] kthread+0x10d/0x150
<4> [178.982915] ? __pfx_kthread+0x10/0x10
<4> [178.982918] ret_from_fork+0x3d4/0x480
<4> [178.982921] ? __pfx_kthread+0x10/0x10
<4> [178.982924] ret_from_fork_asm+0x1a/0x30
<4> [178.982932] </TASK>
<4> [178.982933] irq event stamp: 2517389
<4> [178.982934] hardirqs last enabled at (2517395): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [178.982936] hardirqs last disabled at (2517400): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [178.982939] softirqs last enabled at (2514320): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.982941] softirqs last disabled at (2514313): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [178.982943] ---[ end trace 0000000000000000 ]---
<7> [178.982958] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [178.983063] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [178.983164] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [178.983237] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [178.983243] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [178.983466] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [178.983676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [178.983767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [178.983860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [178.983951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440
<7> [178.984039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [178.984125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [178.984212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [178.984299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100
<7> [178.984381] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [178.984477] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [178.984594] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [178.985629] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [178.995654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [178.995909] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [178.996944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [178.997025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [178.997104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [178.997183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [178.997272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [178.997383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [178.997494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [178.997606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [178.997742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [178.997859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [178.997967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [178.998048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [178.998127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [178.998205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [178.998282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [178.998358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [178.998432] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [178.998507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [178.998583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [178.998676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [178.998759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [178.998839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [178.998921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210
<7> [178.998998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8
<7> [178.999077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc
<7> [178.999157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01
<7> [178.999237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c
<7> [178.999317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [178.999398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [178.999475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [178.999555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [178.999633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [178.999738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [178.999822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [178.999905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100
<7> [178.999986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [179.000066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [179.000143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [179.000219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [179.000294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [179.000374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [179.000456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [179.000536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [179.000615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [179.000708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [179.000785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [179.000867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [179.000948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [179.001029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [179.001107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [179.001184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [179.001264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [179.001342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [179.001419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [179.001495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [179.001572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [179.001655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [179.001735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [179.001816] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [179.001929] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [179.001934] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=55344, lrc_seqno=55344, guc_id=0, flags=0x73 in no process [-1]
<7> [179.001936] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [179.001995] ------------[ cut here ]------------
<4> [179.001996] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [179.001997] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:9/209
<4> [179.002067] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal cmdlinepart intel_powerclamp hid_generic coretemp spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof binfmt_misc kvm_intel usbhid kvm hid irqbypass ghash_clmulni_intel aesni_intel r8169 rapl snd_hda_intel snd_intel_dspcfg snd_hda_codec video intel_cstate snd_hda_core snd_hwdep realtek snd_pcm i2c_i801 snd_timer i2c_mux mei_me snd spi_intel_pci idma64 i2c_smbus spi_intel soundcore mei intel_pmc_core nls_iso8859_1 pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry intel_vsec pinctrl_alderlake wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics fuse
<4> [179.002132] efi_pstore nfnetlink autofs4
<4> [179.002137] CPU: 3 UID: 0 PID: 209 Comm: kworker/u64:9 Tainted: G S U W 7.0.0-lgci-xe-xe-4909-137513b61b4f18795-debug+ #1 PREEMPT(lazy)
<4> [179.002140] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN
<4> [179.002141] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [179.002143] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [179.002148] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4> [179.002217] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 a6 d7 5e e1 48 89 c6 48 8d 3d 3c 94 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4> [179.002219] RSP: 0018:ffffc9000164fca0 EFLAGS: 00010246
<4> [179.002221] RAX: ffffffffa1204543 RBX: 0000000000000000 RCX: 0000000000000000
<4> [179.002222] RDX: ffff8881048d8c90 RSI: ffffffffa1204543 RDI: ffffffffa1003ef0
<4> [179.002224] RBP: ffffc9000164fdb0 R08: 0000000000000000 R09: 0000000000000000
<4> [179.002225] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [179.002226] R13: ffff8881048d8c90 R14: ffff888108743818 R15: 00000000ffffffc2
<4> [179.002227] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000
<4> [179.002229] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [179.002230] CR2: 000077a5bc1e62f4 CR3: 0000000114e1a006 CR4: 0000000000f72ef0
<4> [179.002232] PKRU: 55555554
<4> [179.002233] Call Trace:
<4> [179.002234] <TASK>
<4> [179.002238] ? lock_sync+0x100/0x100
<4> [179.002243] ? lock_release+0xd0/0x2b0
<4> [179.002248] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [179.002254] process_one_work+0x239/0x760
<4> [179.002260] worker_thread+0x200/0x3f0
<4> [179.002263] ? __pfx_worker_thread+0x10/0x10
<4> [179.002265] kthread+0x10d/0x150
<4> [179.002268] ? __pfx_kthread+0x10/0x10
<4> [179.002272] ret_from_fork+0x3d4/0x480
<4> [179.002274] ? __pfx_kthread+0x10/0x10
<4> [179.002277] ret_from_fork_asm+0x1a/0x30
<4> [179.002285] </TASK>
<4> [179.002286] irq event stamp: 2520501
<4> [179.002287] hardirqs last enabled at (2520507): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0
<4> [179.002290] hardirqs last disabled at (2520512): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0
<4> [179.002292] softirqs last enabled at (2519468): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [179.002295] softirqs last disabled at (2519461): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160
<4> [179.002297] ---[ end trace 0000000000000000 ]---
<6> [179.002299] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
|