Results for igt@xe_exec_system_allocator@threads-many-execqueues-mmap-prefetch

Result: Abort 517 Warning(s)

i915_display_info18 igt_runner18 results18.json results18-xe-load.json guc_logs18.tar i915_display_info_post_exec18 boot18 dmesg18

DetailValue
Duration unknown
Hostname
shard-bmg-7
Igt-Version
IGT-Version: 2.3-gc49da3be7 (x86_64) (Linux: 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ x86_64)
Out
Using IGT_SRANDOM=1770698242 for randomisation
Opened device: /dev/dri/card0
Starting subtest: threads-many-execqueues-mmap-prefetch
[thread:2825] Stack trace:
[thread:2825]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2825]   #1 [xe_wait_ufence+0x57]
[thread:2825]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2825]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2825]   #4 [pthread_condattr_setpshared+0x684]
[thread:2825]   #5 [__clone+0x24c]
[thread:2826] Stack trace:
[thread:2827] Stack trace:
[thread:2827]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2826]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2826]   #1 [xe_wait_ufence+0x57]
[thread:2827]   #1 [xe_wait_ufence+0x57]
[thread:2827]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2827]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2826]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2826]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2827]   #4 [pthread_condattr_setpshared+0x684]
[thread:2826]   #4 [pthread_condattr_setpshared+0x684]
[thread:2827]   #5 [__clone+0x24c]
[thread:2826]   #5 [__clone+0x24c]
[thread:2830] Stack trace:
[thread:2830]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2830]   #1 [xe_wait_ufence+0x57]
[thread:2830]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2830]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2831] Stack trace:
[thread:2830]   #4 [pthread_condattr_setpshared+0x684]
[thread:2830]   #5 [__clone+0x24c]
[thread:2831]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2831]   #1 [xe_wait_ufence+0x57]
[thread:2831]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2831]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2831]   #4 [pthread_condattr_setpshared+0x684]
[thread:2831]   #5 [__clone+0x24c]
[thread:2828] Stack trace:
[thread:2828]   #0 ../lib/igt_core.c:2075 __igt_fail_assert()
[thread:2828]   #1 [xe_wait_ufence+0x57]
[thread:2828]   #2 ../tests/intel/xe_exec_system_allocator.c:1732 test_exec()
[thread:2828]   #3 ../tests/intel/xe_exec_system_allocator.c:2019 thread()
[thread:2828]   #4 [pthread_condattr_setpshared+0x684]
[thread:2828]   #5 [__clone+0x24c]
[thread:2829] Pagefault count: before=512, after=73845
runner: This test was killed due to a kernel taint (0x40244).
runner: This test was killed due to exceeding disk usage limit. (Used 10696245 bytes, limit 10485760)

This test caused an abort condition: Kernel badly tainted (0x40244, 0x200) (check dmesg for details):
	TAINT_WARN: WARN_ON has happened.
Err
Starting subtest: threads-many-execqueues-mmap-prefetch
(xe_exec_system_allocator:2823) [thread:2825] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2825] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2825] xe/xe_ioctl-CRITICAL: Last errno: 5, Input/output error
(xe_exec_system_allocator:2823) [thread:2825] xe/xe_ioctl-CRITICAL: error: -5 != 0
(xe_exec_system_allocator:2823) [thread:2826] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2826] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2827] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2827] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2827] xe/xe_ioctl-CRITICAL: Last errno: 5, Input/output error
(xe_exec_system_allocator:2823) [thread:2827] xe/xe_ioctl-CRITICAL: error: -5 != 0
(xe_exec_system_allocator:2823) [thread:2826] xe/xe_ioctl-CRITICAL: Last errno: 5, Input/output error
(xe_exec_system_allocator:2823) [thread:2826] xe/xe_ioctl-CRITICAL: error: -5 != 0
(xe_exec_system_allocator:2823) [thread:2830] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2830] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2830] xe/xe_ioctl-CRITICAL: Last errno: 62, Timer expired
(xe_exec_system_allocator:2823) [thread:2830] xe/xe_ioctl-CRITICAL: error: -62 != 0
(xe_exec_system_allocator:2823) [thread:2831] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2831] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2831] xe/xe_ioctl-CRITICAL: Last errno: 62, Timer expired
(xe_exec_system_allocator:2823) [thread:2831] xe/xe_ioctl-CRITICAL: error: -62 != 0
(xe_exec_system_allocator:2823) [thread:2828] xe/xe_ioctl-CRITICAL: Test assertion failure function xe_wait_ufence, file ../lib/xe/xe_ioctl.c:712:
(xe_exec_system_allocator:2823) [thread:2828] xe/xe_ioctl-CRITICAL: Failed assertion: __xe_wait_ufence(fd, addr, value, exec_queue, &timeout) == 0
(xe_exec_system_allocator:2823) [thread:2828] xe/xe_ioctl-CRITICAL: Last errno: 62, Timer expired
(xe_exec_system_allocator:2823) [thread:2828] xe/xe_ioctl-CRITICAL: error: -62 != 0
Received signal SIGQUIT.
Stack trace: 
 #0 [fatal_sig_handler+0x17b]
 #1 [__sigaction+0x50]
 #2 [__nptl_death_event+0x17f]
 #3 [pthread_join+0x173]
 #4 [threads.constprop.0+0x29f]
 #5 [__igt_unique____real_main2255+0x2052]
 #6 [main+0x2d]
 #7 [__libc_init_first+0x8a]
 #8 [__libc_start_main+0x8b]
 #9 [_start+0x25]
Dmesg

<6> [98.724416] Console: switching to colour dummy device 80x25
<6> [98.724492] [IGT] xe_exec_system_allocator: executing
<6> [98.751345] [IGT] xe_exec_system_allocator: starting subtest threads-many-execqueues-mmap-prefetch
<7> [98.752400] xe 0000:03:00.0: [drm:drm_pagemap_dev_unhold_work [drm_gpusvm_helper]] Releasing reference on provider device and module.
<7> [98.775307] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.776119] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.776423] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.776817] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.777332] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.777589] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778007] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778024] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778100] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778542] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778583] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778629] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778676] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778723] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.778783] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.779608] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.779636] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.781020] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.781033] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.781078] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.781935] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.781972] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.782039] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.783454] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.784315] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.784329] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.784628] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<6> [98.784781] xe 0000:03:00.0: [drm] Tile0: GT0:
ASID: 0
Faulted Address: 0x00000002fb4b7000
FaultType: 0
AccessType: 0
FaultLevel: 2
EngineClass: 3 bcs
EngineInstance: 8
<6> [98.784804] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL
<7> [98.785429] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.785443] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<6> [98.785461] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0
<7> [98.785662] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<6> [98.786506] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249
<5> [98.786536] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4043, lrc_seqno=4043, guc_id=0, flags=0x73 in no process [-1]
<7> [98.786541] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.786661] ------------[ cut here ]------------
<4> [98.786662] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.786664] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#13: kworker/u64:3/203
<4> [98.786783] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.786887] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.786895] CPU: 13 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.786899] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [N]=TEST
<4> [98.786901] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.786903] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.786910] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.787025] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.787028] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.787031] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.787033] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.787034] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.787036] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.787037] R13: ffff8881046a6f10 R14: ffff88812fa00000 R15: 00000000fffffffb
<4> [98.787039] FS: 0000000000000000(0000) GS:ffff8888db35a000(0000) knlGS:0000000000000000
<4> [98.787042] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.787043] CR2: 0000797c95517000 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [98.787045] PKRU: 55555554
<4> [98.787047] Call Trace:
<4> [98.787048] <TASK>
<4> [98.787059] ? __pfx_autoremove_wake_function+0x10/0x10
<4> [98.787069] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.787078] process_one_work+0x22e/0x6b0
<4> [98.787088] worker_thread+0x1e8/0x3d0
<4> [98.787092] ? __pfx_worker_thread+0x10/0x10
<4> [98.787095] kthread+0x11f/0x250
<4> [98.787100] ? __pfx_kthread+0x10/0x10
<4> [98.787105] ret_from_fork+0x344/0x3a0
<4> [98.787108] ? __pfx_kthread+0x10/0x10
<4> [98.787112] ret_from_fork_asm+0x1a/0x30
<4> [98.787123] </TASK>
<4> [98.787125] irq event stamp: 1538363
<4> [98.787126] hardirqs last enabled at (1538369): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.787130] hardirqs last disabled at (1538374): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.787132] softirqs last enabled at (1537796): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.787136] softirqs last disabled at (1537791): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.787139] ---[ end trace 0000000000000000 ]---
<6> [98.787141] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.787253] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.787329] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.787397] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.787884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.787976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.788067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.788153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.788233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.788313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.788394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.788474] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.788565] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.788679] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.789758] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.799828] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.800083] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.801731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.801812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.801904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.801980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.802054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.802126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.802199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.802271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.802343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.802416] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.802488] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.802560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.802632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.802703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.802775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.802855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.802925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.802995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.803063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.803142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.803217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.803293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.803368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.803442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.803515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.803591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.803670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.803746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.803827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.803902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.803977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.804057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.804138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.804218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.804295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.804370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.804443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.804517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.804589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.804665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.804738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.804811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.804898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.804975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.805049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.805121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.805193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.805267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.805339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.805414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.805492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.805566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.805640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.805711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.805785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.805867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.805938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.806014] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.806730] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.806735] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4043, lrc_seqno=4043, guc_id=0, flags=0x73 in no process [-1]
<7> [98.806737] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.806796] ------------[ cut here ]------------
<4> [98.806797] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.806798] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:3/203
<4> [98.806881] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.806940] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.806945] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.806948] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.806949] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.806950] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.806955] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.807022] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.807024] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.807026] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.807027] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.807029] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.807030] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.807031] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.807032] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [98.807033] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.807034] CR2: 0000797c95511000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [98.807036] PKRU: 55555554
<4> [98.807037] Call Trace:
<4> [98.807038] <TASK>
<4> [98.807043] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.807048] ? lock_release+0xce/0x280
<4> [98.807054] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.807059] process_one_work+0x22e/0x6b0
<4> [98.807065] worker_thread+0x1e8/0x3d0
<4> [98.807067] ? __pfx_worker_thread+0x10/0x10
<4> [98.807070] kthread+0x11f/0x250
<4> [98.807073] ? __pfx_kthread+0x10/0x10
<4> [98.807076] ret_from_fork+0x344/0x3a0
<4> [98.807078] ? __pfx_kthread+0x10/0x10
<4> [98.807081] ret_from_fork_asm+0x1a/0x30
<4> [98.807087] </TASK>
<4> [98.807088] irq event stamp: 1542567
<4> [98.807089] hardirqs last enabled at (1542573): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.807092] hardirqs last disabled at (1542578): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.807095] softirqs last enabled at (1541272): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.807097] softirqs last disabled at (1541265): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.807099] ---[ end trace 0000000000000000 ]---
<6> [98.807101] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.807172] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.807845] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.808826] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.808930] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.809104] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.809379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.809465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.809555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.809639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.809723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.809805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.809932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.810015] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.810104] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.810217] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.811245] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.821826] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.822081] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.823672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.823799] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.823927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.824045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.824162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.824278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.824394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.824509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.824625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.824741] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.824862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.824978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.825094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.825210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.825325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.825441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.825557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.825672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.825788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.825917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.826039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.826159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.826280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.826399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.826517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.826639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.826765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.826892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.827014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.827136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.827258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.827385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.827512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.827638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.827763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.827890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.828011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.828131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.828250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.828373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.828494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.828615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.828733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.828861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.828981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.829102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.829224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.829346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.829465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.829586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.829709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.829833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.829953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.830072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.830193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.830312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.830430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.830553] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.830938] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.830944] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4043, lrc_seqno=4043, guc_id=0, flags=0x73 in no process [-1]
<7> [98.830946] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.831008] ------------[ cut here ]------------
<4> [98.831009] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.831011] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:3/203
<4> [98.831085] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.831142] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.831147] CPU: 0 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.831150] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.831151] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.831152] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.831157] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.831226] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.831228] RSP: 0000:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.831230] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.831231] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.831233] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.831234] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.831235] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.831236] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [98.831237] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.831238] CR2: 0000797c7c000020 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [98.831240] PKRU: 55555554
<4> [98.831241] Call Trace:
<4> [98.831242] <TASK>
<4> [98.831247] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.831252] ? lock_release+0xce/0x280
<4> [98.831257] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.831262] process_one_work+0x22e/0x6b0
<4> [98.831268] worker_thread+0x1e8/0x3d0
<4> [98.831270] ? __pfx_worker_thread+0x10/0x10
<4> [98.831273] kthread+0x11f/0x250
<4> [98.831276] ? __pfx_kthread+0x10/0x10
<4> [98.831279] ret_from_fork+0x344/0x3a0
<4> [98.831281] ? __pfx_kthread+0x10/0x10
<4> [98.831284] ret_from_fork_asm+0x1a/0x30
<4> [98.831290] </TASK>
<4> [98.831291] irq event stamp: 1547233
<4> [98.831292] hardirqs last enabled at (1547239): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.831295] hardirqs last disabled at (1547244): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.831297] softirqs last enabled at (1545670): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.831299] softirqs last disabled at (1545661): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.831302] ---[ end trace 0000000000000000 ]---
<7> [98.832398] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.832412] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.832424] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.832454] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.832467] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [98.832616] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4054, lrc_seqno=4054, guc_id=0, flags=0x73 in no process [-1]
<7> [98.832619] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.832681] ------------[ cut here ]------------
<4> [98.832682] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.832683] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:3/203
<4> [98.832752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.832810] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.832815] CPU: 0 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.832825] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.832827] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.832828] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.832833] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.832900] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.832901] RSP: 0000:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.832903] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.832904] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.832906] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.832907] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.832908] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.832909] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [98.832910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.832911] CR2: 0000797c7c000020 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [98.832912] PKRU: 55555554
<4> [98.832913] Call Trace:
<4> [98.832914] <TASK>
<4> [98.832919] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.832924] ? lock_release+0xce/0x280
<4> [98.832929] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.832934] process_one_work+0x22e/0x6b0
<4> [98.832939] worker_thread+0x1e8/0x3d0
<4> [98.832942] ? __pfx_worker_thread+0x10/0x10
<4> [98.832944] kthread+0x11f/0x250
<4> [98.832947] ? __pfx_kthread+0x10/0x10
<4> [98.832950] ret_from_fork+0x344/0x3a0
<4> [98.832952] ? __pfx_kthread+0x10/0x10
<4> [98.832955] ret_from_fork_asm+0x1a/0x30
<4> [98.832961] </TASK>
<4> [98.832962] irq event stamp: 1548159
<4> [98.832963] hardirqs last enabled at (1548165): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.832966] hardirqs last disabled at (1548170): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.832968] softirqs last enabled at (1547968): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.832970] softirqs last disabled at (1547959): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.832972] ---[ end trace 0000000000000000 ]---
<6> [98.832974] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.833041] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.833047] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.833062] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.833179] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.833302] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.833570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.833656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.833744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.833837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.833951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.834034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.834114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.834193] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.834282] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.834392] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.835406] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.845824] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.846082] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.847403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.847472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.847537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.847604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.847669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.847734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.847798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.847894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.847986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.848056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.848128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.848200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.848273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.848344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.848413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.848482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.848553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.848623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.848691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.848769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.848859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.848942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.849018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.849091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.849167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.849246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.849327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.849403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.849477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.849552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.849626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.849706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.849786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.849878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.849955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.850032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.850104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.850177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.850249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.850325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.850401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.850478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.850551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.850629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.850702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.850776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.850857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.850932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.851003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.851074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.851149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.851222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.851294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.851366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.851439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.851513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.851585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.851661] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.852105] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.852110] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4054, lrc_seqno=4054, guc_id=0, flags=0x73 in no process [-1]
<7> [98.852113] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.852171] ------------[ cut here ]------------
<4> [98.852172] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.852173] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#11: kworker/u64:10/210
<4> [98.852243] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.852302] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.852307] CPU: 11 UID: 0 PID: 210 Comm: kworker/u64:10 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.852310] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.852311] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.852312] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.852317] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.852384] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.852385] RSP: 0018:ffffc9000164bc98 EFLAGS: 00010246
<4> [98.852387] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.852389] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.852390] RBP: ffffc9000164bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.852391] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.852392] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.852393] FS: 0000000000000000(0000) GS:ffff8888db25a000(0000) knlGS:0000000000000000
<4> [98.852394] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.852396] CR2: 00007c07f65e06f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [98.852397] PKRU: 55555554
<4> [98.852398] Call Trace:
<4> [98.852399] <TASK>
<4> [98.852404] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.852409] ? lock_release+0xce/0x280
<4> [98.852414] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.852419] process_one_work+0x22e/0x6b0
<4> [98.852425] worker_thread+0x1e8/0x3d0
<4> [98.852428] ? __pfx_worker_thread+0x10/0x10
<4> [98.852430] kthread+0x11f/0x250
<4> [98.852433] ? __pfx_kthread+0x10/0x10
<4> [98.852436] ret_from_fork+0x344/0x3a0
<4> [98.852438] ? __pfx_kthread+0x10/0x10
<4> [98.852441] ret_from_fork_asm+0x1a/0x30
<4> [98.852448] </TASK>
<4> [98.852449] irq event stamp: 508741
<4> [98.852450] hardirqs last enabled at (508747): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.852452] hardirqs last disabled at (508752): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.852455] softirqs last enabled at (507192): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.852457] softirqs last disabled at (507185): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.852460] ---[ end trace 0000000000000000 ]---
<6> [98.852461] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.852530] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.852536] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.852954] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.853221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.853302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.853386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.853467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.853546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.853628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.853712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.853790] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.853893] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.854010] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.855022] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.864824] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.865079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.866309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.866384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.866456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.866529] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.866601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.866674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.866747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.866833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.866904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.866974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.867044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.867115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.867184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.867253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.867323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.867392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.867461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.867530] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.867600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.867677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.867754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.867838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.867916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.867989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.868063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.868139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.868217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.868293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.868367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.868441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.868516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.868595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.868675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.868757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.868839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.868915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.868987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.869059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.869130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.869205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.869278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.869350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.869424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.869501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.869573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.869644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.869716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.869794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.869880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.869951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.870028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.870102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.870176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.870248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.870322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.870396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.870467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.870542] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.870991] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.871835] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4054, lrc_seqno=4054, guc_id=0, flags=0x73 in no process [-1]
<7> [98.871841] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.871945] ------------[ cut here ]------------
<4> [98.871947] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.871948] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#12: kworker/u64:10/210
<7> [98.871973] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [98.872063] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep
<7> [98.872073] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [98.872121] intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.872155] CPU: 12 UID: 0 PID: 210 Comm: kworker/u64:10 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.872159] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.872160] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.872161] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.872173] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.872284] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.872286] RSP: 0018:ffffc9000164bc98 EFLAGS: 00010246
<4> [98.872289] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.872291] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.872293] RBP: ffffc9000164bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.872295] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.872296] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.872298] FS: 0000000000000000(0000) GS:ffff8888db2da000(0000) knlGS:0000000000000000
<4> [98.872300] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.872302] CR2: 0000797c95541000 CR3: 000000015481e005 CR4: 0000000000f72ef0
<4> [98.872304] PKRU: 55555554
<4> [98.872306] Call Trace:
<4> [98.872307] <TASK>
<4> [98.872316] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.872323] ? lock_release+0xce/0x280
<4> [98.872332] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.872340] process_one_work+0x22e/0x6b0
<4> [98.872350] worker_thread+0x1e8/0x3d0
<4> [98.872353] ? __pfx_worker_thread+0x10/0x10
<4> [98.872357] kthread+0x11f/0x250
<4> [98.872361] ? __pfx_kthread+0x10/0x10
<4> [98.872366] ret_from_fork+0x344/0x3a0
<4> [98.872368] ? __pfx_kthread+0x10/0x10
<4> [98.872372] ret_from_fork_asm+0x1a/0x30
<4> [98.872383] </TASK>
<4> [98.872385] irq event stamp: 513307
<4> [98.872386] hardirqs last enabled at (513313): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.872389] hardirqs last disabled at (513318): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.872392] softirqs last enabled at (511760): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.872395] softirqs last disabled at (511751): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.872398] ---[ end trace 0000000000000000 ]---
<7> [98.873216] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.873223] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.873234] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.873374] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.873387] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.873400] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [98.873497] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4066, lrc_seqno=4066, guc_id=0, flags=0x73 in no process [-1]
<7> [98.873502] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.873564] ------------[ cut here ]------------
<4> [98.873565] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.873566] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:3/203
<4> [98.873639] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.873696] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.873701] CPU: 7 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.873704] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.873705] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.873706] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.873711] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.873780] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.873781] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.873783] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.873785] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.873786] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.873787] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.873788] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.873789] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [98.873790] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.873792] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [98.873793] PKRU: 55555554
<4> [98.873794] Call Trace:
<4> [98.873795] <TASK>
<4> [98.873800] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.873805] ? lock_release+0xce/0x280
<4> [98.873810] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.873826] process_one_work+0x22e/0x6b0
<4> [98.873835] worker_thread+0x1e8/0x3d0
<4> [98.873838] ? __pfx_worker_thread+0x10/0x10
<4> [98.873840] kthread+0x11f/0x250
<4> [98.873843] ? __pfx_kthread+0x10/0x10
<4> [98.873847] ret_from_fork+0x344/0x3a0
<4> [98.873849] ? __pfx_kthread+0x10/0x10
<4> [98.873852] ret_from_fork_asm+0x1a/0x30
<4> [98.873858] </TASK>
<4> [98.873859] irq event stamp: 1550533
<4> [98.873860] hardirqs last enabled at (1550539): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.873863] hardirqs last disabled at (1550544): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.873865] softirqs last enabled at (1550474): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.873868] softirqs last disabled at (1550465): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.873870] ---[ end trace 0000000000000000 ]---
<6> [98.873872] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.873941] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.873949] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.874002] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.874347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.874479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.874614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.874743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.874912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.875040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.875169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.875296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.875434] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.875596] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.877000] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.886823] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.887140] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.888555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.888673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.888787] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.888913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.889030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.889146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.889262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.889378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.889494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.889610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.889726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.889858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.889975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.890091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.890207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.890323] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.890438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.890554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.890670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.890791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.890917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.891038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.891159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.891278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.891397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.891519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.891645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.891768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.891894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.892016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.892138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.892266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.892392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.892519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.892644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.892767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.892893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.893014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.893133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.893256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.893378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.893498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.893617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.893742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.893867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.893987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.894109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.894232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.894352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.894472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.894597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.894718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.894846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.894966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.895087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.895206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.895325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.895448] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.895907] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.895912] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4066, lrc_seqno=4066, guc_id=0, flags=0x73 in no process [-1]
<7> [98.895914] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.895972] ------------[ cut here ]------------
<4> [98.895973] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.895975] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:10/210
<4> [98.896041] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.896097] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.896102] CPU: 7 UID: 0 PID: 210 Comm: kworker/u64:10 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.896104] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.896105] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.896106] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.896111] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.896178] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.896180] RSP: 0018:ffffc9000164bc98 EFLAGS: 00010246
<4> [98.896182] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.896183] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.896184] RBP: ffffc9000164bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.896185] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.896186] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.896187] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [98.896189] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.896190] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [98.896191] PKRU: 55555554
<4> [98.896192] Call Trace:
<4> [98.896193] <TASK>
<4> [98.896198] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.896202] ? lock_release+0xce/0x280
<4> [98.896208] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.896213] process_one_work+0x22e/0x6b0
<4> [98.896218] worker_thread+0x1e8/0x3d0
<4> [98.896221] ? __pfx_worker_thread+0x10/0x10
<4> [98.896223] kthread+0x11f/0x250
<4> [98.896226] ? __pfx_kthread+0x10/0x10
<4> [98.896229] ret_from_fork+0x344/0x3a0
<4> [98.896231] ? __pfx_kthread+0x10/0x10
<4> [98.896234] ret_from_fork_asm+0x1a/0x30
<4> [98.896240] </TASK>
<4> [98.896241] irq event stamp: 518695
<4> [98.896242] hardirqs last enabled at (518701): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.896245] hardirqs last disabled at (518706): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.896247] softirqs last enabled at (517138): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.896249] softirqs last disabled at (517131): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.896251] ---[ end trace 0000000000000000 ]---
<6> [98.896253] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.896318] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.896325] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.897622] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.897723] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.897845] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.898116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.898203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.898293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.898377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.898459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.898539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.898621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.898700] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.898792] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.898920] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.899930] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.909820] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.910079] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.911411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.911486] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.911557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.911631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.911703] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.911776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.911863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.911938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.912009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.912081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.912150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.912220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.912289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.912358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.912427] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.912495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.912564] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.912633] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.912702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.912780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.912868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.912942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.913017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.913089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.913162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.913237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.913314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.913389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.913462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.913537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.913612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.913696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.913778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.913871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.913948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.914025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.914097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.914170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.914242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.914316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.914389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.914461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.914533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.914608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.914680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.914753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.914836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.914911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.914983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.915054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.915128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.915202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.915274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.915347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.915421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.915494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.915568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.915645] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.915978] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.916832] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4066, lrc_seqno=4066, guc_id=0, flags=0x73 in no process [-1]
<7> [98.916835] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.916894] ------------[ cut here ]------------
<4> [98.916895] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.916896] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:7/207
<4> [98.916966] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.917024] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.917029] CPU: 6 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.917032] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.917033] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.917034] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.917039] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.917107] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.917109] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [98.917111] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.917112] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.917113] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.917115] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.917116] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.917117] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [98.917118] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.917119] CR2: 0000797c9551f098 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [98.917121] PKRU: 55555554
<4> [98.917122] Call Trace:
<4> [98.917123] <TASK>
<4> [98.917128] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.917133] ? lock_release+0xce/0x280
<4> [98.917138] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.917143] process_one_work+0x22e/0x6b0
<4> [98.917149] worker_thread+0x1e8/0x3d0
<4> [98.917152] ? __pfx_worker_thread+0x10/0x10
<4> [98.917154] kthread+0x11f/0x250
<4> [98.917157] ? __pfx_kthread+0x10/0x10
<4> [98.917160] ret_from_fork+0x344/0x3a0
<4> [98.917162] ? __pfx_kthread+0x10/0x10
<4> [98.917165] ret_from_fork_asm+0x1a/0x30
<4> [98.917171] </TASK>
<4> [98.917172] irq event stamp: 172583
<4> [98.917173] hardirqs last enabled at (172589): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.917176] hardirqs last disabled at (172594): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.917178] softirqs last enabled at (171130): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.917181] softirqs last disabled at (171123): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.917183] ---[ end trace 0000000000000000 ]---
<7> [98.917673] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.917677] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.917683] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [98.917696] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4079, lrc_seqno=4079, guc_id=0, flags=0x73 in no process [-1]
<7> [98.917703] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.917699] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.917767] ------------[ cut here ]------------
<4> [98.917768] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.917769] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:7/207
<4> [98.917879] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1
<7> [98.917944] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [98.917948] pmt_discovery
<4> [98.917949] pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.917971] CPU: 6 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.917974] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.917975] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.917976] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.917980] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.918405] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.918407] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [98.918409] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.918410] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.918411] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.918412] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.918413] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.918414] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [98.918416] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.918417] CR2: 0000797c9551f098 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [98.918418] PKRU: 55555554
<4> [98.918419] Call Trace:
<4> [98.918420] <TASK>
<4> [98.918425] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.918429] ? lock_release+0xce/0x280
<4> [98.918435] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.918440] process_one_work+0x22e/0x6b0
<4> [98.918445] worker_thread+0x1e8/0x3d0
<4> [98.918448] ? __pfx_worker_thread+0x10/0x10
<4> [98.918450] kthread+0x11f/0x250
<4> [98.918453] ? __pfx_kthread+0x10/0x10
<4> [98.918457] ret_from_fork+0x344/0x3a0
<4> [98.918459] ? __pfx_kthread+0x10/0x10
<4> [98.918462] ret_from_fork_asm+0x1a/0x30
<4> [98.918468] </TASK>
<4> [98.918469] irq event stamp: 173825
<4> [98.918470] hardirqs last enabled at (173831): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.918472] hardirqs last disabled at (173836): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.918474] softirqs last enabled at (173076): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.918477] softirqs last disabled at (173071): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.918479] ---[ end trace 0000000000000000 ]---
<6> [98.918481] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.918551] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.918559] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.918578] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.918692] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.918800] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.919475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.919621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.919767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.919916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.920053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.920185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.920318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.920450] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.920593] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.920767] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.921782] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.931821] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.932078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.932992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.933068] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.933141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.933214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.933286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.933358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.933429] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.933498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.933567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.933636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.933706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.933776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.933863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.933936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.934007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.934078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.934147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.934217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.934284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.934362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.934439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.934513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.934590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.934664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.934737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.934816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.934896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.934971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.935045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.935120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.935195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.935275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.935357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.935437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.935514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.935589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.935663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.935738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.935816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.935890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.935964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.936037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.936109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.936185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.936258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.936330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.936403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.936476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.936547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.936619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.936694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.936767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.936853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.936927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.937002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.937073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.937145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.937222] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.937548] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.938353] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4079, lrc_seqno=4079, guc_id=0, flags=0x73 in no process [-1]
<7> [98.938356] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.938415] ------------[ cut here ]------------
<4> [98.938416] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.938417] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:3/203
<4> [98.938493] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.938551] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.938556] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.938559] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.938560] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.938561] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.938566] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.938634] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.938635] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.938637] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.938639] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.938640] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.938641] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.938642] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.938643] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [98.938644] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.938645] CR2: 0000797c9551f098 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [98.938647] PKRU: 55555554
<4> [98.938648] Call Trace:
<4> [98.938649] <TASK>
<4> [98.938654] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.938659] ? lock_release+0xce/0x280
<4> [98.938664] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.938669] process_one_work+0x22e/0x6b0
<4> [98.938675] worker_thread+0x1e8/0x3d0
<4> [98.938677] ? __pfx_worker_thread+0x10/0x10
<4> [98.938680] kthread+0x11f/0x250
<4> [98.938683] ? __pfx_kthread+0x10/0x10
<4> [98.938686] ret_from_fork+0x344/0x3a0
<4> [98.938688] ? __pfx_kthread+0x10/0x10
<4> [98.938691] ret_from_fork_asm+0x1a/0x30
<4> [98.938697] </TASK>
<4> [98.938698] irq event stamp: 1556405
<4> [98.938699] hardirqs last enabled at (1556411): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.938702] hardirqs last disabled at (1556416): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.938704] softirqs last enabled at (1555090): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.938706] softirqs last disabled at (1555083): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.938709] ---[ end trace 0000000000000000 ]---
<6> [98.938710] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.938779] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<7> [98.938795] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.938926] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [98.939018] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.939108] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.939371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.939456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.939543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.939625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.939708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.939789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.939884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.939965] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.940057] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.940167] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.941174] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.951816] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.952072] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.952991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.953067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.953139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.953212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.953284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.953355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.953425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.953495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.953566] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.953639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.953710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.953781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.953862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.953933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.954002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.954071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.954142] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.954210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.954281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.954357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.954434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.954508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.954583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.954656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.954730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.954807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.954897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.954973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.955051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.955127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.955202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.955284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.955364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.955444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.955526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.955601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.955675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.955748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.955824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.955901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.955974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.956047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.956120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.956198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.956270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.956344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.956417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.956489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.956560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.956631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.956705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.956778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.956859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.956934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.957008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.957079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.957154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.957230] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.957546] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.957550] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4079, lrc_seqno=4079, guc_id=0, flags=0x73 in no process [-1]
<7> [98.957552] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.957609] ------------[ cut here ]------------
<4> [98.957610] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.957611] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:3/203
<4> [98.957682] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.957739] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.957744] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.957746] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.957747] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.957748] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.957753] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.957824] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.957826] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.957829] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.957830] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.957831] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.957832] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.957834] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.957835] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [98.957837] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.957838] CR2: 0000797c9551f098 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [98.957839] PKRU: 55555554
<4> [98.957840] Call Trace:
<4> [98.957841] <TASK>
<4> [98.957846] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.957851] ? lock_release+0xce/0x280
<4> [98.957857] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.957862] process_one_work+0x22e/0x6b0
<4> [98.957867] worker_thread+0x1e8/0x3d0
<4> [98.957870] ? __pfx_worker_thread+0x10/0x10
<4> [98.957872] kthread+0x11f/0x250
<4> [98.957875] ? __pfx_kthread+0x10/0x10
<4> [98.957879] ret_from_fork+0x344/0x3a0
<4> [98.957881] ? __pfx_kthread+0x10/0x10
<4> [98.957884] ret_from_fork_asm+0x1a/0x30
<4> [98.957890] </TASK>
<4> [98.957891] irq event stamp: 1560893
<4> [98.957892] hardirqs last enabled at (1560899): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.957895] hardirqs last disabled at (1560904): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.957897] softirqs last enabled at (1559278): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.957900] softirqs last disabled at (1559273): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.957902] ---[ end trace 0000000000000000 ]---
<7> [98.958285] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [98.958861] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [98.958922] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4090, lrc_seqno=4090, guc_id=0, flags=0x73 in no process [-1]
<7> [98.958925] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.958994] ------------[ cut here ]------------
<4> [98.958995] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.958996] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:3/203
<4> [98.959075] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.959164] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.959174] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.959177] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.959178] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.959180] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.959185] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.959263] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.959264] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [98.959266] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.959267] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.959268] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.959269] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.959270] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.959272] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [98.959273] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.959274] CR2: 0000797c9551f098 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [98.959275] PKRU: 55555554
<4> [98.959276] Call Trace:
<4> [98.959277] <TASK>
<4> [98.959282] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.959287] ? lock_release+0xce/0x280
<4> [98.959292] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.959297] process_one_work+0x22e/0x6b0
<4> [98.959302] worker_thread+0x1e8/0x3d0
<4> [98.959305] ? __pfx_worker_thread+0x10/0x10
<4> [98.959307] kthread+0x11f/0x250
<4> [98.959310] ? __pfx_kthread+0x10/0x10
<4> [98.959314] ret_from_fork+0x344/0x3a0
<4> [98.959316] ? __pfx_kthread+0x10/0x10
<4> [98.959319] ret_from_fork_asm+0x1a/0x30
<4> [98.959325] </TASK>
<4> [98.959326] irq event stamp: 1561803
<4> [98.959327] hardirqs last enabled at (1561809): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.959329] hardirqs last disabled at (1561814): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.959331] softirqs last enabled at (1559278): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.959334] softirqs last disabled at (1559273): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.959336] ---[ end trace 0000000000000000 ]---
<6> [98.959337] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.959406] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.959413] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.959441] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.959557] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.959671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.960036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.960170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.960304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.960433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.960561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.960687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.960824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.960951] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.961088] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.961247] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.962612] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.972819] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.973136] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.974120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.974235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.974349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.974465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.974581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.974696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.974822] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.974942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.975060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.975177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.975292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.975408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.975523] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.975639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.975754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.975877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.975994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [98.976109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [98.976225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [98.976345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [98.976465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [98.976584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [98.976705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [98.976829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [98.976948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [98.977069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [98.977195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [98.977317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [98.977438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [98.977559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [98.977681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [98.977812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [98.977939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [98.978065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [98.978189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [98.978313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [98.978434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [98.978554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [98.978673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [98.978796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [98.978923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [98.979044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [98.979163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [98.979287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [98.979406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [98.979526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [98.979645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [98.979765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [98.979888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [98.980007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [98.980130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [98.980250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [98.980369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [98.980488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [98.980609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [98.980728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [98.980850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [98.980974] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [98.981355] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [98.982534] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4090, lrc_seqno=4090, guc_id=0, flags=0x73 in no process [-1]
<7> [98.982538] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [98.982639] ------------[ cut here ]------------
<4> [98.982640] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [98.982642] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#15: kworker/u64:7/207
<4> [98.982750] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [98.982847] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [98.982854] CPU: 15 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [98.982858] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [98.982860] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [98.982861] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [98.982869] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [98.982976] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [98.982978] RSP: 0000:ffffc90001573c98 EFLAGS: 00010246
<4> [98.982981] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [98.982983] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [98.982985] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [98.982986] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [98.982988] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [98.982989] FS: 0000000000000000(0000) GS:ffff8888db45a000(0000) knlGS:0000000000000000
<4> [98.982991] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [98.982993] CR2: 0000797c95542098 CR3: 0000000194f43003 CR4: 0000000000f72ef0
<4> [98.982995] PKRU: 55555554
<4> [98.982996] Call Trace:
<4> [98.982998] <TASK>
<4> [98.983007] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [98.983014] ? lock_release+0xce/0x280
<4> [98.983023] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [98.983031] process_one_work+0x22e/0x6b0
<4> [98.983041] worker_thread+0x1e8/0x3d0
<4> [98.983044] ? __pfx_worker_thread+0x10/0x10
<4> [98.983047] kthread+0x11f/0x250
<4> [98.983052] ? __pfx_kthread+0x10/0x10
<4> [98.983057] ret_from_fork+0x344/0x3a0
<4> [98.983059] ? __pfx_kthread+0x10/0x10
<4> [98.983063] ret_from_fork_asm+0x1a/0x30
<4> [98.983074] </TASK>
<4> [98.983075] irq event stamp: 179159
<4> [98.983077] hardirqs last enabled at (179165): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [98.983080] hardirqs last disabled at (179170): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [98.983083] softirqs last enabled at (178748): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.983086] softirqs last disabled at (178735): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [98.983089] ---[ end trace 0000000000000000 ]---
<6> [98.983091] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [98.983199] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [98.983208] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [98.983267] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [98.983413] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [98.983857] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [98.984199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [98.984325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [98.984456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [98.984583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [98.984710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [98.984842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [98.984970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [98.985096] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [98.985232] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [98.985390] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [98.986767] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [98.996818] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [98.997134] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [98.998113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [98.998230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [98.998344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [98.998460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [98.998575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [98.998690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [98.998814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [98.998930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [98.999046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [98.999161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [98.999277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [98.999392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [98.999508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [98.999623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [98.999739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [98.999859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [98.999975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.000091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.000206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.000326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.000446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.000565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.000686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.000809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.000929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.001050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.001175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.001298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.001419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.001540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.001662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.001789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.001920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.002047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.002171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.002294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.002415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.002535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.002653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.002777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.002902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.003022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.003141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.003265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.003385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.003504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.003623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.003744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.003868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.003987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.004109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.004229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.004349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.004468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.004589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.004707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.004829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.004952] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.005295] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.005814] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4090, lrc_seqno=4090, guc_id=0, flags=0x73 in no process [-1]
<7> [99.005819] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.005913] ------------[ cut here ]------------
<4> [99.005914] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.005915] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:7/207
<4> [99.005984] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.006048] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.006054] CPU: 6 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.006058] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.006058] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.006060] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.006065] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.006127] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.006129] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.006131] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.006132] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.006133] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.006134] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.006135] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.006136] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.006137] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.006138] CR2: 0000797c9551f098 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.006140] PKRU: 55555554
<4> [99.006140] Call Trace:
<4> [99.006141] <TASK>
<4> [99.006146] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.006150] ? lock_release+0xce/0x280
<4> [99.006155] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.006159] process_one_work+0x22e/0x6b0
<4> [99.006165] worker_thread+0x1e8/0x3d0
<4> [99.006167] ? __pfx_worker_thread+0x10/0x10
<4> [99.006169] kthread+0x11f/0x250
<4> [99.006172] ? __pfx_kthread+0x10/0x10
<4> [99.006175] ret_from_fork+0x344/0x3a0
<4> [99.006177] ? __pfx_kthread+0x10/0x10
<4> [99.006180] ret_from_fork_asm+0x1a/0x30
<4> [99.006186] </TASK>
<4> [99.006187] irq event stamp: 183667
<4> [99.006188] hardirqs last enabled at (183673): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.006191] hardirqs last disabled at (183678): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.006193] softirqs last enabled at (182322): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.006195] softirqs last disabled at (182317): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.006198] ---[ end trace 0000000000000000 ]---
<7> [99.006238] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.006374] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [99.006532] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4098, lrc_seqno=4098, guc_id=0, flags=0x73 in no process [-1]
<7> [99.006552] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.006535] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.006575] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.006580] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.006603] ------------[ cut here ]------------
<4> [99.006605] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [99.006623] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.006607] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:7/207
<4> [99.006685] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.006741] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.006745] CPU: 6 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.006747] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.006748] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.006749] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.006754] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.006837] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.006839] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.006841] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.006842] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.006843] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.006844] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.006845] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.006846] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.006848] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.006849] CR2: 0000797c9551f098 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.006850] PKRU: 55555554
<4> [99.006851] Call Trace:
<4> [99.006852] <TASK>
<4> [99.006857] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.006861] ? lock_release+0xce/0x280
<4> [99.006867] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.006872] process_one_work+0x22e/0x6b0
<4> [99.006878] worker_thread+0x1e8/0x3d0
<4> [99.006881] ? __pfx_worker_thread+0x10/0x10
<4> [99.006883] kthread+0x11f/0x250
<4> [99.006886] ? __pfx_kthread+0x10/0x10
<4> [99.006890] ret_from_fork+0x344/0x3a0
<4> [99.006892] ? __pfx_kthread+0x10/0x10
<4> [99.006895] ret_from_fork_asm+0x1a/0x30
<4> [99.006901] </TASK>
<4> [99.006902] irq event stamp: 184691
<4> [99.006903] hardirqs last enabled at (184697): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.006906] hardirqs last disabled at (184702): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.006908] softirqs last enabled at (184530): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.006910] softirqs last disabled at (184475): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.006912] ---[ end trace 0000000000000000 ]---
<6> [99.006914] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.006983] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.006990] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.007041] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.007309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.007395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.007484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.007568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.007651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.007734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.007830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.007910] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.008002] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.008112] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.009119] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.019847] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.020193] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.021067] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.021148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.021224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.021299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.021370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.021441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.021512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.021583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.021653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.021723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.021792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.021898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.021974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.022044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.022116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.022185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.022255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.022328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.022400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.022480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.022558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.022633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.022709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.022784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.022870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.022946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.023024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.023100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.023173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.023247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.023320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.023400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.023479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.023558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.023635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.023709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.023785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.023871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.023943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.024019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.024094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.024170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.024246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.024322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.024393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.024465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.024537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.024611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.024682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.024753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.024833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.024907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.024978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.025048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.025120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.025189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.025260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.025335] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.025638] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.025643] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4098, lrc_seqno=4098, guc_id=0, flags=0x73 in no process [-1]
<7> [99.025645] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.025702] ------------[ cut here ]------------
<4> [99.025703] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.025705] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:1/118
<4> [99.025774] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.025839] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.025845] CPU: 10 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.025848] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.025849] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.025850] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.025855] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.025922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.025924] RSP: 0000:ffffc9000054bc98 EFLAGS: 00010246
<4> [99.025926] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.025927] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.025928] RBP: ffffc9000054bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.025930] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.025931] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.025932] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.025933] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.025934] CR2: 0000797c955261d8 CR3: 0000000194f43002 CR4: 0000000000f72ef0
<4> [99.025936] PKRU: 55555554
<4> [99.025937] Call Trace:
<4> [99.025938] <TASK>
<4> [99.025943] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.025948] ? lock_release+0xce/0x280
<4> [99.025953] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.025958] process_one_work+0x22e/0x6b0
<4> [99.025964] worker_thread+0x1e8/0x3d0
<4> [99.025967] ? __pfx_worker_thread+0x10/0x10
<4> [99.025969] kthread+0x11f/0x250
<4> [99.025972] ? __pfx_kthread+0x10/0x10
<4> [99.025975] ret_from_fork+0x344/0x3a0
<4> [99.025977] ? __pfx_kthread+0x10/0x10
<4> [99.025980] ret_from_fork_asm+0x1a/0x30
<4> [99.025986] </TASK>
<4> [99.025987] irq event stamp: 310935
<4> [99.025988] hardirqs last enabled at (310941): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.025991] hardirqs last disabled at (310946): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.025993] softirqs last enabled at (309410): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.025995] softirqs last disabled at (309403): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.025997] ---[ end trace 0000000000000000 ]---
<6> [99.025999] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.026068] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.026878] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.026982] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.027101] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.027203] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.027467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.027551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.027639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.027723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.027812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.027894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.027976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.028055] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.028147] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.028260] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.029408] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.039811] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.040064] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.040914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.040989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.041061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.041134] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.041207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.041277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.041348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.041418] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.041487] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.041557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.041627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.041696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.041767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.041850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.041920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.041988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.042060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.042128] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.042197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.042279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.042356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.042431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.042508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.042585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.042661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.042739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.042823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.042900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.042974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.043050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.043126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.043206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.043285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.043364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.043440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.043516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.043590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.043664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.043736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.043812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.043886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.043961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.044034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.044110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.044183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.044255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.044327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.044403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.044479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.044551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.044627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.044702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.044776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.044859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.044936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.045011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.045085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.045161] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.045457] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.045461] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4098, lrc_seqno=4098, guc_id=0, flags=0x73 in no process [-1]
<7> [99.045463] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.045521] ------------[ cut here ]------------
<4> [99.045522] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.045524] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:1/118
<4> [99.045593] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.045651] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.045655] CPU: 10 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.045658] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.045659] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.045660] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.045665] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.045732] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.045733] RSP: 0000:ffffc9000054bc98 EFLAGS: 00010246
<4> [99.045735] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.045736] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.045738] RBP: ffffc9000054bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.045739] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.045740] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.045741] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.045742] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.045744] CR2: 0000797c955261d8 CR3: 0000000194f43002 CR4: 0000000000f72ef0
<4> [99.045745] PKRU: 55555554
<4> [99.045746] Call Trace:
<4> [99.045747] <TASK>
<4> [99.045752] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.045757] ? lock_release+0xce/0x280
<4> [99.045762] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.045767] process_one_work+0x22e/0x6b0
<4> [99.045773] worker_thread+0x1e8/0x3d0
<4> [99.045775] ? __pfx_worker_thread+0x10/0x10
<4> [99.045777] kthread+0x11f/0x250
<4> [99.045781] ? __pfx_kthread+0x10/0x10
<4> [99.045784] ret_from_fork+0x344/0x3a0
<4> [99.045786] ? __pfx_kthread+0x10/0x10
<4> [99.045789] ret_from_fork_asm+0x1a/0x30
<4> [99.045795] </TASK>
<4> [99.045796] irq event stamp: 315423
<4> [99.045797] hardirqs last enabled at (315429): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.045799] hardirqs last disabled at (315434): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.045805] softirqs last enabled at (314118): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.045808] softirqs last disabled at (314107): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.045810] ---[ end trace 0000000000000000 ]---
<7> [99.046335] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.046411] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.047081] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4108, lrc_seqno=4108, guc_id=0, flags=0x73 in no process [-1]
<7> [99.047084] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.047181] ------------[ cut here ]------------
<4> [99.047182] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.047183] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:1/118
<4> [99.047252] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.047308] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.047312] CPU: 10 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.047314] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.047315] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.047316] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.047321] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.047389] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.047390] RSP: 0000:ffffc9000054bc98 EFLAGS: 00010246
<4> [99.047392] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.047393] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.047394] RBP: ffffc9000054bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.047395] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.047397] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.047398] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.047399] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.047400] CR2: 0000797c955261d8 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.047401] PKRU: 55555554
<4> [99.047402] Call Trace:
<4> [99.047403] <TASK>
<4> [99.047408] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.047413] ? lock_release+0xce/0x280
<4> [99.047418] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.047423] process_one_work+0x22e/0x6b0
<4> [99.047428] worker_thread+0x1e8/0x3d0
<4> [99.047431] ? __pfx_worker_thread+0x10/0x10
<4> [99.047433] kthread+0x11f/0x250
<4> [99.047436] ? __pfx_kthread+0x10/0x10
<4> [99.047439] ret_from_fork+0x344/0x3a0
<4> [99.047441] ? __pfx_kthread+0x10/0x10
<4> [99.047444] ret_from_fork_asm+0x1a/0x30
<4> [99.047450] </TASK>
<4> [99.047451] irq event stamp: 316333
<4> [99.047452] hardirqs last enabled at (316339): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.047455] hardirqs last disabled at (316344): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.047457] softirqs last enabled at (314118): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.047459] softirqs last disabled at (314107): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.047461] ---[ end trace 0000000000000000 ]---
<6> [99.047463] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.047531] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.047537] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.047549] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.047647] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.047749] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.048019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.048108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.048199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.048284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.048366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.048448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.048529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.048609] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.048702] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.048819] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.049831] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.059808] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.060064] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.060941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.061015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.061088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.061162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.061235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.061307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.061378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.061448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.061519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.061589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.061660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.061730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.061820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.061893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.061965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.062037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.062107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.062177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.062248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.062327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.062404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.062479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.062556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.062631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.062704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.062782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.062875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.062952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.063027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.063101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.063176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.063255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.063333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.063411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.063488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.063565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.063638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.063712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.063784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.063869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.063944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.064017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.064088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.064164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.064240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.064313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.064385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.064459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.064530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.064602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.064677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.064750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.064834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.064907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.064980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.065051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.065123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.065200] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.065549] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.065554] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4108, lrc_seqno=4108, guc_id=0, flags=0x73 in no process [-1]
<7> [99.065557] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.065618] ------------[ cut here ]------------
<4> [99.065619] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.065620] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.065693] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.065750] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.065755] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.065758] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.065759] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.065760] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.065765] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.065868] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.065870] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.065874] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.065877] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.065879] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.065881] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.065883] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.065885] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.065887] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.065890] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.065892] PKRU: 55555554
<4> [99.065894] Call Trace:
<4> [99.065896] <TASK>
<4> [99.065905] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.065913] ? lock_release+0xce/0x280
<4> [99.065922] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.065931] process_one_work+0x22e/0x6b0
<4> [99.065942] worker_thread+0x1e8/0x3d0
<4> [99.065946] ? __pfx_worker_thread+0x10/0x10
<4> [99.065950] kthread+0x11f/0x250
<4> [99.065955] ? __pfx_kthread+0x10/0x10
<4> [99.065961] ret_from_fork+0x344/0x3a0
<4> [99.065963] ? __pfx_kthread+0x10/0x10
<4> [99.065968] ret_from_fork_asm+0x1a/0x30
<4> [99.065980] </TASK>
<4> [99.065982] irq event stamp: 1567917
<4> [99.065983] hardirqs last enabled at (1567923): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.065988] hardirqs last disabled at (1567928): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.065991] softirqs last enabled at (1567756): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.065994] softirqs last disabled at (1567747): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.065998] ---[ end trace 0000000000000000 ]---
<6> [99.066000] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.066078] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.066083] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.066998] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.067107] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.067208] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.067474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.067557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.067647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.067730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.067814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.067893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.067973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.068050] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.068141] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.068253] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.069259] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.079835] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.080090] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.080949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.081024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.081097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.081171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.081243] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.081314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.081385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.081454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.081524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.081594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.081664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.081734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.081807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.081878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.081947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.082016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.082086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.082156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.082225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.082302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.082379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.082454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.082531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.082604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.082678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.082755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.082846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.082922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.082998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.083074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.083148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.083230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.083312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.083393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.083470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.083547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.083620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.083693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.083766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.083848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.083922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.083995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.084065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.084143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.084215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.084288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.084361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.084435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.084507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.084580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.084656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.084731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.084810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.084884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.084958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.085030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.085101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.085175] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.085498] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.085502] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4108, lrc_seqno=4108, guc_id=0, flags=0x73 in no process [-1]
<7> [99.085504] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.085561] ------------[ cut here ]------------
<4> [99.085562] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.085563] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:1/118
<4> [99.085633] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.085690] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.085695] CPU: 10 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.085697] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.085698] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.085699] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.085704] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.085772] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.085773] RSP: 0000:ffffc9000054bc98 EFLAGS: 00010246
<4> [99.085775] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.085777] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.085778] RBP: ffffc9000054bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.085779] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.085780] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.085781] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.085782] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.085784] CR2: 0000797c955261d8 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.085785] PKRU: 55555554
<4> [99.085786] Call Trace:
<4> [99.085787] <TASK>
<4> [99.085792] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.085797] ? lock_release+0xce/0x280
<4> [99.085806] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.085811] process_one_work+0x22e/0x6b0
<4> [99.085817] worker_thread+0x1e8/0x3d0
<4> [99.085820] ? __pfx_worker_thread+0x10/0x10
<4> [99.085822] kthread+0x11f/0x250
<4> [99.085825] ? __pfx_kthread+0x10/0x10
<4> [99.085829] ret_from_fork+0x344/0x3a0
<4> [99.085831] ? __pfx_kthread+0x10/0x10
<4> [99.085834] ret_from_fork_asm+0x1a/0x30
<4> [99.085840] </TASK>
<4> [99.085841] irq event stamp: 321449
<4> [99.085842] hardirqs last enabled at (321455): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.085845] hardirqs last disabled at (321460): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.085847] softirqs last enabled at (321384): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.085849] softirqs last disabled at (321379): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.085852] ---[ end trace 0000000000000000 ]---
<7> [99.086194] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.086238] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.086342] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.086376] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.086894] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4116, lrc_seqno=4116, guc_id=0, flags=0x73 in no process [-1]
<7> [99.086897] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.086991] ------------[ cut here ]------------
<4> [99.086992] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.086993] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:1/118
<4> [99.087062] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.087118] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.087122] CPU: 10 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.087124] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.087125] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.087126] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.087131] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.087198] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.087200] RSP: 0000:ffffc9000054bc98 EFLAGS: 00010246
<4> [99.087201] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.087203] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.087204] RBP: ffffc9000054bda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.087205] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.087206] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.087207] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.087208] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.087209] CR2: 0000797c955261d8 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.087210] PKRU: 55555554
<4> [99.087211] Call Trace:
<4> [99.087212] <TASK>
<4> [99.087217] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.087221] ? lock_release+0xce/0x280
<4> [99.087227] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.087232] process_one_work+0x22e/0x6b0
<4> [99.087237] worker_thread+0x1e8/0x3d0
<4> [99.087240] ? __pfx_worker_thread+0x10/0x10
<4> [99.087242] kthread+0x11f/0x250
<4> [99.087245] ? __pfx_kthread+0x10/0x10
<4> [99.087249] ret_from_fork+0x344/0x3a0
<4> [99.087251] ? __pfx_kthread+0x10/0x10
<4> [99.087253] ret_from_fork_asm+0x1a/0x30
<4> [99.087260] </TASK>
<4> [99.087260] irq event stamp: 322345
<4> [99.087261] hardirqs last enabled at (322351): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.087264] hardirqs last disabled at (322356): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.087266] softirqs last enabled at (321384): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.087268] softirqs last disabled at (321379): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.087270] ---[ end trace 0000000000000000 ]---
<6> [99.087272] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.087340] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.087346] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.087359] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.087479] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.087596] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.087871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.087967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.088056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.088140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.088223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.088305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.088385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.088464] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.088558] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.088671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.089685] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.099808] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.100065] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.100969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.101047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.101121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.101194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.101266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.101337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.101410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.101482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.101553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.101623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.101695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.101765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.101866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.101943] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.102012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.102083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.102152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.102221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.102290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.102368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.102445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.102519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.102596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.102668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.102739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.102822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.102909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.102985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.103058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.103132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.103205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.103285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.103365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.103446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.103521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.103596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.103669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.103742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.103819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.103906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.103980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.104054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.104126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.104202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.104274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.104346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.104417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.104491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.104561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.104631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.104706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.104778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.104864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.104946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.105019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.105089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.105160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.105234] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.105546] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.105550] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4116, lrc_seqno=4116, guc_id=0, flags=0x73 in no process [-1]
<7> [99.105552] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.105609] ------------[ cut here ]------------
<4> [99.105610] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.105611] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.105683] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.105741] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.105746] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.105749] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.105750] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.105751] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.105756] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.105836] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.105838] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.105842] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.105844] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.105846] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.105848] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.105850] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.105852] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.105855] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.105857] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.105860] PKRU: 55555554
<4> [99.105861] Call Trace:
<4> [99.105863] <TASK>
<4> [99.105872] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.105880] ? lock_release+0xce/0x280
<4> [99.105889] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.105899] process_one_work+0x22e/0x6b0
<4> [99.105909] worker_thread+0x1e8/0x3d0
<4> [99.105913] ? __pfx_worker_thread+0x10/0x10
<4> [99.105917] kthread+0x11f/0x250
<4> [99.105922] ? __pfx_kthread+0x10/0x10
<4> [99.105928] ret_from_fork+0x344/0x3a0
<4> [99.105931] ? __pfx_kthread+0x10/0x10
<4> [99.105935] ret_from_fork_asm+0x1a/0x30
<4> [99.105947] </TASK>
<4> [99.105949] irq event stamp: 1573401
<4> [99.105951] hardirqs last enabled at (1573407): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.105955] hardirqs last disabled at (1573412): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.105958] softirqs last enabled at (1571882): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.105961] softirqs last disabled at (1571875): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.105965] ---[ end trace 0000000000000000 ]---
<6> [99.105968] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.106042] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.106860] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.106971] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.107070] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.107175] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.107440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.107526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.107613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.107696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.107778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.107875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.107964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.108044] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.108132] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.108244] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.109253] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.119807] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.120061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.120892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.120969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.121042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.121115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.121186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.121256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.121326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.121397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.121466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.121537] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.121608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.121679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.121750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.121830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.121912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.121985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.122056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.122126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.122199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.122276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.122351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.122426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.122503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.122577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.122650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.122726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.122815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.122904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.122982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.123058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.123135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.123214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.123294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.123373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.123451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.123526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.123597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.123669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.123742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.123825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.123909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.123982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.124054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.124129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.124202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.124274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.124344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.124417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.124488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.124558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.124632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.124704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.124779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.124870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.124956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.125028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.125101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.125176] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.125507] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.125511] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4116, lrc_seqno=4116, guc_id=0, flags=0x73 in no process [-1]
<7> [99.125514] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.125571] ------------[ cut here ]------------
<4> [99.125572] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.125573] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.125641] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.125699] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.125703] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.125706] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.125707] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.125708] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.125713] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.125780] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.125782] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.125783] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.125785] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.125786] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.125787] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.125788] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.125789] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.125790] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.125791] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.125793] PKRU: 55555554
<4> [99.125794] Call Trace:
<4> [99.125795] <TASK>
<4> [99.125807] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.125815] ? lock_release+0xce/0x280
<4> [99.125823] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.125832] process_one_work+0x22e/0x6b0
<4> [99.125843] worker_thread+0x1e8/0x3d0
<4> [99.125847] ? __pfx_worker_thread+0x10/0x10
<4> [99.125851] kthread+0x11f/0x250
<4> [99.125856] ? __pfx_kthread+0x10/0x10
<4> [99.125861] ret_from_fork+0x344/0x3a0
<4> [99.125864] ? __pfx_kthread+0x10/0x10
<4> [99.125868] ret_from_fork_asm+0x1a/0x30
<4> [99.125879] </TASK>
<4> [99.125881] irq event stamp: 1577933
<4> [99.125883] hardirqs last enabled at (1577939): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.125887] hardirqs last disabled at (1577944): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.125890] softirqs last enabled at (1576626): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.125894] softirqs last disabled at (1576609): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.125897] ---[ end trace 0000000000000000 ]---
<7> [99.126424] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.126433] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.126804] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4126, lrc_seqno=4126, guc_id=0, flags=0x73 in no process [-1]
<7> [99.126808] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.126881] ------------[ cut here ]------------
<4> [99.126882] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.126883] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.126952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.127009] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.127014] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.127016] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.127017] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.127018] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.127023] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.127090] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.127092] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.127094] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.127095] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.127096] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.127097] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.127098] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.127099] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.127100] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.127102] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.127103] PKRU: 55555554
<4> [99.127104] Call Trace:
<4> [99.127105] <TASK>
<4> [99.127107] ? trace_hardirqs_on+0x63/0xd0
<4> [99.127113] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.127117] ? lock_release+0xce/0x280
<4> [99.127123] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.127128] process_one_work+0x22e/0x6b0
<4> [99.127133] worker_thread+0x1e8/0x3d0
<4> [99.127136] ? __pfx_worker_thread+0x10/0x10
<4> [99.127138] kthread+0x11f/0x250
<4> [99.127141] ? __pfx_kthread+0x10/0x10
<4> [99.127144] ret_from_fork+0x344/0x3a0
<4> [99.127146] ? __pfx_kthread+0x10/0x10
<4> [99.127149] ret_from_fork_asm+0x1a/0x30
<4> [99.127155] </TASK>
<4> [99.127156] irq event stamp: 1578855
<4> [99.127157] hardirqs last enabled at (1578861): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.127159] hardirqs last disabled at (1578866): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.127161] softirqs last enabled at (1578056): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.127164] softirqs last disabled at (1578049): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.127166] ---[ end trace 0000000000000000 ]---
<6> [99.127167] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.127234] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.127242] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.127268] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.127369] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.127472] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.127746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.127853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.127944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.128028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.128111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.128194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.128276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.128357] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.128451] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.128563] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.129636] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.140357] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.140613] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.141496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.141574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.141664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.141739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.141819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.141891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.141961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.142032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.142104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.142174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.142244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.142314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.142384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.142456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.142526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.142597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.142667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.142737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.142816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.142895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.142972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.143048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.143124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.143198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.143272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.143351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.143429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.143505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.143578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.143653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.143730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.143816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.143897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.143979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.144060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.144136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.144210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.144284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.144357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.144432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.144505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.144577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.144649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.144725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.144800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.144873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.144944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.145018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.145090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.145162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.145236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.145309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.145387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.145461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.145535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.145606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.145678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.145751] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.146047] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.146825] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4126, lrc_seqno=4126, guc_id=0, flags=0x73 in no process [-1]
<7> [99.146830] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.146893] ------------[ cut here ]------------
<4> [99.146894] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.146896] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:0/12
<4> [99.146974] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg
<7> [99.146952] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.147009] ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.147038] CPU: 7 UID: 0 PID: 12 Comm: kworker/u64:0 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.147041] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.147042] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.147043] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.147048] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.147118] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<7> [99.147068] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.147120] RSP: 0018:ffffc900000b7c98 EFLAGS: 00010246
<4> [99.147122] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.147123] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.147124] RBP: ffffc900000b7da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.147125] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.147126] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.147127] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.147128] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.147130] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.147131] PKRU: 55555554
<4> [99.147132] Call Trace:
<4> [99.147132] <TASK>
<4> [99.147138] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.147142] ? lock_release+0xce/0x280
<4> [99.147148] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.147153] process_one_work+0x22e/0x6b0
<4> [99.147159] worker_thread+0x1e8/0x3d0
<4> [99.147161] ? __pfx_worker_thread+0x10/0x10
<4> [99.147163] kthread+0x11f/0x250
<4> [99.147166] ? __pfx_kthread+0x10/0x10
<4> [99.147170] ret_from_fork+0x344/0x3a0
<4> [99.147172] ? __pfx_kthread+0x10/0x10
<4> [99.147175] ret_from_fork_asm+0x1a/0x30
<4> [99.147181] </TASK>
<4> [99.147182] irq event stamp: 22615
<4> [99.147183] hardirqs last enabled at (22621): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.147186] hardirqs last disabled at (22626): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.147188] softirqs last enabled at (20930): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.147190] softirqs last disabled at (20921): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.147193] ---[ end trace 0000000000000000 ]---
<6> [99.147195] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.147263] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.147270] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.147319] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.147585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.147673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.147761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.147883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.147964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.148046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.148127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.148206] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.148296] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.148408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.149418] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.159804] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.160061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.160941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.161017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.161088] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.161177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.161254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.161326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.161397] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.161468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.161538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.161609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.161679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.161749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.161828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.161897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.161966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.162037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.162109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.162181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.162253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.162333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.162409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.162483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.162557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.162632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.162705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.162781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.162878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.162955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.163030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.163106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.163182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.163264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.163343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.163421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.163497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.163571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.163644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.163718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.163790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.163883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.163961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.164035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.164108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.164185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.164259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.164334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.164406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.164479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.164549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.164621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.164697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.164769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.164853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.164927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.165001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.165072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.165144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.165220] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.165518] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.165824] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4126, lrc_seqno=4126, guc_id=0, flags=0x73 in no process [-1]
<7> [99.165834] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.166204] ------------[ cut here ]------------
<4> [99.166207] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.166210] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:0/12
<4> [99.166366] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.166432] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.166442] CPU: 6 UID: 0 PID: 12 Comm: kworker/u64:0 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.166447] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.166449] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.166451] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [99.166438] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.166460] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.166597] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.166600] RSP: 0018:ffffc900000b7c98 EFLAGS: 00010246
<4> [99.166603] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.166606] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.166608] RBP: ffffc900000b7da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.166611] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.166614] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.166616] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.166619] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.166621] CR2: 0000797c9551f098 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.166624] PKRU: 55555554
<4> [99.166626] Call Trace:
<4> [99.166628] <TASK>
<4> [99.166638] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.166646] ? lock_release+0xce/0x280
<4> [99.166655] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.166666] process_one_work+0x22e/0x6b0
<4> [99.166678] worker_thread+0x1e8/0x3d0
<4> [99.166683] ? __pfx_worker_thread+0x10/0x10
<4> [99.166687] kthread+0x11f/0x250
<7> [99.166615] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.166694] ? __pfx_kthread+0x10/0x10
<4> [99.166699] ret_from_fork+0x344/0x3a0
<4> [99.166701] ? __pfx_kthread+0x10/0x10
<4> [99.166705] ret_from_fork_asm+0x1a/0x30
<4> [99.166715] </TASK>
<4> [99.166716] irq event stamp: 27105
<4> [99.166718] hardirqs last enabled at (27111): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.166721] hardirqs last disabled at (27116): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.166724] softirqs last enabled at (25792): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.166726] softirqs last disabled at (25787): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.166729] ---[ end trace 0000000000000000 ]---
<5> [99.166979] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4134, lrc_seqno=4134, guc_id=0, flags=0x73 in no process [-1]
<7> [99.166982] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.167011] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.167041] ------------[ cut here ]------------
<4> [99.167042] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.167043] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:0/12
<7> [99.167103] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.167113] Modules linked in:
<7> [99.167110] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.167114] snd_hda_codec_intelhdmi
<4> [99.167115] snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper
<7> [99.167125] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.167128] cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.167177] CPU: 6 UID: 0 PID: 12 Comm: kworker/u64:0 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.167180] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.167181] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.167182] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.167186] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.167253] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.167254] RSP: 0018:ffffc900000b7c98 EFLAGS: 00010246
<4> [99.167256] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.167257] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.167258] RBP: ffffc900000b7da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.167259] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.167260] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.167261] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.167263] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.167264] CR2: 0000797c9551f098 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.167265] PKRU: 55555554
<4> [99.167266] Call Trace:
<4> [99.167267] <TASK>
<4> [99.167272] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.167276] ? lock_release+0xce/0x280
<4> [99.167282] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.167287] process_one_work+0x22e/0x6b0
<4> [99.167292] worker_thread+0x1e8/0x3d0
<4> [99.167295] ? __pfx_worker_thread+0x10/0x10
<4> [99.167297] kthread+0x11f/0x250
<4> [99.167300] ? __pfx_kthread+0x10/0x10
<4> [99.167303] ret_from_fork+0x344/0x3a0
<4> [99.167305] ? __pfx_kthread+0x10/0x10
<4> [99.167308] ret_from_fork_asm+0x1a/0x30
<4> [99.167314] </TASK>
<4> [99.167315] irq event stamp: 28147
<4> [99.167316] hardirqs last enabled at (28153): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.167318] hardirqs last disabled at (28158): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.167320] softirqs last enabled at (25792): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.167323] softirqs last disabled at (25787): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.167325] ---[ end trace 0000000000000000 ]---
<6> [99.167327] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.167393] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.167401] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.167476] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.167906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.167996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.168097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.168183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.168266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.168347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.168430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.168509] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.168602] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.168727] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.169741] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.179804] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.180061] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.180926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.181003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.181076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.181150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.181223] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.181293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.181364] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.181435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.181505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.181577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.181647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.181717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.181786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.181880] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.181955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.182028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.182098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.182168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.182239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.182318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.182395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.182471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.182547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.182621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.182693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.182769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.182859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.182936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.183011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.183087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.183162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.183242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.183323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.183403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.183480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.183555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.183629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.183702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.183776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.183872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.183966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.184046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.184116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.184194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.184267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.184341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.184413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.184489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.184562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.184635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.184727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.184868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.184946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.185022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.185098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.185171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.185244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.185323] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.185613] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.185825] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4134, lrc_seqno=4134, guc_id=0, flags=0x73 in no process [-1]
<7> [99.185833] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.186188] ------------[ cut here ]------------
<4> [99.186190] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.186192] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:8/208
<4> [99.186438] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.186497] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.186502] CPU: 0 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.186505] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.186506] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.186507] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.186512] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<7> [99.186530] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.186623] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.186626] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.186629] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.186630] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.186632] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.186633] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.186634] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.186636] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.186638] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.186640] CR2: 0000797c955341d8 CR3: 0000000194f43003 CR4: 0000000000f72ef0
<4> [99.186642] PKRU: 55555554
<4> [99.186643] Call Trace:
<4> [99.186645] <TASK>
<4> [99.186654] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.186662] ? lock_release+0xce/0x280
<4> [99.186671] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.186681] process_one_work+0x22e/0x6b0
<4> [99.186691] worker_thread+0x1e8/0x3d0
<4> [99.186695] ? __pfx_worker_thread+0x10/0x10
<4> [99.186698] kthread+0x11f/0x250
<4> [99.186702] ? __pfx_kthread+0x10/0x10
<4> [99.186707] ret_from_fork+0x344/0x3a0
<4> [99.186709] ? __pfx_kthread+0x10/0x10
<4> [99.186713] ret_from_fork_asm+0x1a/0x30
<4> [99.186724] </TASK>
<4> [99.186726] irq event stamp: 523895
<4> [99.186728] hardirqs last enabled at (523901): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.186732] hardirqs last disabled at (523906): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.186735] softirqs last enabled at (522488): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.186738] softirqs last disabled at (522445): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.186742] ---[ end trace 0000000000000000 ]---
<7> [99.186681] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [99.186744] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.186859] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.186865] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.186913] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.187179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.187266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.187355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.187438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.187521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.187603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.187685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.187762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.187865] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.187978] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.188984] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.198806] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.199063] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.199921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.199999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.200070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.200145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.200217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.200289] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.200360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.200430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.200500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.200573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.200645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.200715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.200786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.200873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.200944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.201013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.201082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.201151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.201219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.201298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.201373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.201447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.201522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.201599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.201674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.201750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.201839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.201915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.201988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.202061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.202139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.202216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.202294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.202371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.202446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.202520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.202593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.202666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.202738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.202820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.202893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.202966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.203036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.203112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.203184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.203256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.203328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.203403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.203475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.203562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.203644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.203720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.203800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.203874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.203949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.204022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.204093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.204226] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.204523] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.204819] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4134, lrc_seqno=4134, guc_id=0, flags=0x73 in no process [-1]
<7> [99.204828] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.205187] ------------[ cut here ]------------
<4> [99.205190] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.205193] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#1: kworker/u64:8/208
<4> [99.205367] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.205428] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.205436] CPU: 1 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.205440] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.205442] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.205444] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.205452] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<7> [99.205444] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.205568] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.205570] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.205573] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.205575] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.205577] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.205579] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.205580] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.205582] FS: 0000000000000000(0000) GS:ffff8888dad5a000(0000) knlGS:0000000000000000
<4> [99.205584] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.205586] CR2: 0000797c955421d8 CR3: 0000000194f43006 CR4: 0000000000f72ef0
<4> [99.205588] PKRU: 55555554
<4> [99.205590] Call Trace:
<4> [99.205594] <TASK>
<4> [99.205603] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.205611] ? lock_release+0xce/0x280
<4> [99.205619] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.205627] process_one_work+0x22e/0x6b0
<4> [99.205637] worker_thread+0x1e8/0x3d0
<4> [99.205642] ? __pfx_worker_thread+0x10/0x10
<4> [99.205646] kthread+0x11f/0x250
<4> [99.205651] ? __pfx_kthread+0x10/0x10
<4> [99.205656] ret_from_fork+0x344/0x3a0
<4> [99.205659] ? __pfx_kthread+0x10/0x10
<7> [99.205595] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.205663] ret_from_fork_asm+0x1a/0x30
<4> [99.205675] </TASK>
<4> [99.205676] irq event stamp: 528371
<4> [99.205677] hardirqs last enabled at (528377): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.205680] hardirqs last disabled at (528382): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.205683] softirqs last enabled at (526856): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.205686] softirqs last disabled at (526851): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.205689] ---[ end trace 0000000000000000 ]---
<5> [99.206051] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4144, lrc_seqno=4144, guc_id=0, flags=0x73 in no process [-1]
<7> [99.206078] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.206117] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.206129] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.206148] ------------[ cut here ]------------
<4> [99.206149] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.206150] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.206223] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.206282] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.206287] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.206289] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.206290] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.206291] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.206296] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.206365] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.206366] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.206369] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.206370] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.206371] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.206372] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.206373] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.206374] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.206376] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.206377] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.206378] PKRU: 55555554
<4> [99.206379] Call Trace:
<4> [99.206380] <TASK>
<4> [99.206385] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.206390] ? lock_release+0xce/0x280
<4> [99.206396] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.206401] process_one_work+0x22e/0x6b0
<4> [99.206406] worker_thread+0x1e8/0x3d0
<4> [99.206409] ? __pfx_worker_thread+0x10/0x10
<4> [99.206411] kthread+0x11f/0x250
<4> [99.206414] ? __pfx_kthread+0x10/0x10
<4> [99.206418] ret_from_fork+0x344/0x3a0
<4> [99.206420] ? __pfx_kthread+0x10/0x10
<4> [99.206422] ret_from_fork_asm+0x1a/0x30
<4> [99.206429] </TASK>
<4> [99.206430] irq event stamp: 1581765
<4> [99.206431] hardirqs last enabled at (1581771): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.206433] hardirqs last disabled at (1581776): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.206435] softirqs last enabled at (1579104): [<ffffffff813346d3>] kernel_fpu_end+0x53/0x70
<4> [99.206438] softirqs last disabled at (1579102): [<ffffffff81334db4>] kernel_fpu_begin_mask+0xc4/0x120
<4> [99.206440] ---[ end trace 0000000000000000 ]---
<6> [99.206442] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.206512] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.206522] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.206572] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.206844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.206929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.207018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.207100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.207182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.207263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.207344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.207424] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.207517] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.207628] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.208639] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.219358] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.219664] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.220604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.220682] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.220754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.220841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.220914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.220985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.221055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.221125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.221196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.221266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.221335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.221406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.221475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.221544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.221614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.221683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.221754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.221836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.221907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.221986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.222063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.222141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.222218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.222291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.222364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.222440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.222517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.222591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.222664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.222738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.222819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.222897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.222975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.223055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.223131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.223208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.223282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.223354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.223425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.223500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.223574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.223647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.223721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.223805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.223880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.223953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.224026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.224101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.224175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.224247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.224322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.224395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.224467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.224539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.224614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.224684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.224756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.224849] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.225150] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.225154] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4144, lrc_seqno=4144, guc_id=0, flags=0x73 in no process [-1]
<7> [99.225156] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.225213] ------------[ cut here ]------------
<4> [99.225214] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.225215] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.225284] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.225341] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.225346] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.225349] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.225350] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.225351] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.225356] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.225424] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.225426] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.225428] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.225429] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.225430] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.225431] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.225432] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.225433] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.225435] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.225436] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.225437] PKRU: 55555554
<4> [99.225438] Call Trace:
<4> [99.225439] <TASK>
<4> [99.225444] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.225449] ? lock_release+0xce/0x280
<4> [99.225455] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.225460] process_one_work+0x22e/0x6b0
<4> [99.225465] worker_thread+0x1e8/0x3d0
<4> [99.225468] ? __pfx_worker_thread+0x10/0x10
<4> [99.225470] kthread+0x11f/0x250
<4> [99.225473] ? __pfx_kthread+0x10/0x10
<4> [99.225477] ret_from_fork+0x344/0x3a0
<4> [99.225479] ? __pfx_kthread+0x10/0x10
<4> [99.225481] ret_from_fork_asm+0x1a/0x30
<4> [99.225488] </TASK>
<4> [99.225489] irq event stamp: 1586271
<4> [99.225490] hardirqs last enabled at (1586277): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.225493] hardirqs last disabled at (1586282): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.225495] softirqs last enabled at (1584792): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.225497] softirqs last disabled at (1584785): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.225499] ---[ end trace 0000000000000000 ]---
<6> [99.225501] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.225568] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.225574] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.226507] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.226651] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.226802] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.227073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.227179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.227268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.227351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.227433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.227512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.227592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.227673] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.227767] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.227891] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.228899] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.238809] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.239066] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.239936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.240013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.240086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.240160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.240234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.240307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.240379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.240449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.240518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.240589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.240658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.240727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.240802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.240873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.240942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.241012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.241081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.241152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.241221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.241299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.241377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.241454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.241535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.241610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.241684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.241759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.241850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.241926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.242010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.242087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.242163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.242244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.242323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.242402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.242520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.242621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.242700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.242774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.242861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.242940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.243014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.243088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.243161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.243239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.243311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.243385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.243458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.243535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.243607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.243678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.243753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.243838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.243911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.243983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.244057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.244129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.244210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.244286] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.244616] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.244620] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4144, lrc_seqno=4144, guc_id=0, flags=0x73 in no process [-1]
<7> [99.244622] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.244681] ------------[ cut here ]------------
<4> [99.244682] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.244683] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#1: kworker/u64:8/208
<4> [99.244752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.244815] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.244820] CPU: 1 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.244823] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.244824] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.244826] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.244831] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.244899] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.244901] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.244903] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.244904] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.244906] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.244907] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.244908] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.244909] FS: 0000000000000000(0000) GS:ffff8888dad5a000(0000) knlGS:0000000000000000
<4> [99.244910] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.244911] CR2: 0000797c955421d8 CR3: 0000000003448001 CR4: 0000000000f72ef0
<4> [99.244913] PKRU: 55555554
<4> [99.244914] Call Trace:
<4> [99.244915] <TASK>
<4> [99.244920] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.244925] ? lock_release+0xce/0x280
<4> [99.244930] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.244935] process_one_work+0x22e/0x6b0
<4> [99.244941] worker_thread+0x1e8/0x3d0
<4> [99.244944] ? __pfx_worker_thread+0x10/0x10
<4> [99.244946] kthread+0x11f/0x250
<4> [99.244949] ? __pfx_kthread+0x10/0x10
<4> [99.244953] ret_from_fork+0x344/0x3a0
<4> [99.244954] ? __pfx_kthread+0x10/0x10
<4> [99.244957] ret_from_fork_asm+0x1a/0x30
<4> [99.244964] </TASK>
<4> [99.244964] irq event stamp: 533911
<4> [99.244965] hardirqs last enabled at (533917): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.244968] hardirqs last disabled at (533922): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.244970] softirqs last enabled at (532398): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.244973] softirqs last disabled at (532393): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.244975] ---[ end trace 0000000000000000 ]---
<7> [99.245309] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.245424] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.245458] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.245546] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.245810] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4152, lrc_seqno=4152, guc_id=0, flags=0x73 in no process [-1]
<7> [99.245815] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.245889] ------------[ cut here ]------------
<4> [99.245890] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.245891] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:8/208
<4> [99.245964] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.246021] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.246026] CPU: 6 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.246029] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.246030] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.246031] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [99.245998] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.246036] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.246104] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.246106] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.246108] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.246109] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.246110] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.246111] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.246112] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.246114] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.246115] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.246116] CR2: 0000797c95549278 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.246118] PKRU: 55555554
<4> [99.246119] Call Trace:
<4> [99.246120] <TASK>
<4> [99.246125] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.246129] ? lock_release+0xce/0x280
<4> [99.246135] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.246139] process_one_work+0x22e/0x6b0
<4> [99.246145] worker_thread+0x1e8/0x3d0
<7> [99.246102] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.246148] ? __pfx_worker_thread+0x10/0x10
<4> [99.246150] kthread+0x11f/0x250
<4> [99.246153] ? __pfx_kthread+0x10/0x10
<4> [99.246157] ret_from_fork+0x344/0x3a0
<4> [99.246158] ? __pfx_kthread+0x10/0x10
<4> [99.246161] ret_from_fork_asm+0x1a/0x30
<4> [99.246167] </TASK>
<4> [99.246168] irq event stamp: 534805
<4> [99.246169] hardirqs last enabled at (534811): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.246172] hardirqs last disabled at (534816): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.246174] softirqs last enabled at (532398): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.246176] softirqs last disabled at (532393): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.246178] ---[ end trace 0000000000000000 ]---
<6> [99.246179] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.246247] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.246253] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.246302] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.246567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.246653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.246742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.246836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.246919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.247000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.247082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.247160] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.247249] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.247360] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.248368] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.258797] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.259052] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.259926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.260009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.260087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.260164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.260238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.260309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.260380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.260451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.260521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.260591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.260661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.260730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.260805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.260876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.260946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.261016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.261085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.261156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.261225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.261303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.261381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.261456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.261530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.261602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.261674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.261749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.261871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.261952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.262026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.262103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.262180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.262261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.262340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.262420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.262496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.262570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.262644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.262717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.262795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.262878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.262956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.263030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.263103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.263179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.263252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.263324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.263398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.263472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.263543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.263614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.263691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.263768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.263854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.263927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.264001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.264073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.264145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.264222] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.264520] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.264524] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4152, lrc_seqno=4152, guc_id=0, flags=0x73 in no process [-1]
<7> [99.264526] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.264584] ------------[ cut here ]------------
<4> [99.264585] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.264586] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.264655] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.264713] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.264718] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.264721] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.264722] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.264723] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.264728] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.264801] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.264803] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.264805] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.264807] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.264808] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.264809] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.264810] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.264812] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.264813] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.264815] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.264816] PKRU: 55555554
<4> [99.264817] Call Trace:
<4> [99.264818] <TASK>
<4> [99.264823] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.264829] ? lock_release+0xce/0x280
<4> [99.264834] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.264839] process_one_work+0x22e/0x6b0
<4> [99.264845] worker_thread+0x1e8/0x3d0
<4> [99.264848] ? __pfx_worker_thread+0x10/0x10
<4> [99.264850] kthread+0x11f/0x250
<4> [99.264854] ? __pfx_kthread+0x10/0x10
<4> [99.264857] ret_from_fork+0x344/0x3a0
<4> [99.264859] ? __pfx_kthread+0x10/0x10
<4> [99.264862] ret_from_fork_asm+0x1a/0x30
<4> [99.264869] </TASK>
<4> [99.264870] irq event stamp: 539493
<4> [99.264871] hardirqs last enabled at (539499): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.264874] hardirqs last disabled at (539504): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.264876] softirqs last enabled at (538082): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.264879] softirqs last disabled at (538075): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.264881] ---[ end trace 0000000000000000 ]---
<6> [99.264883] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.264951] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.265755] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.265855] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.265953] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.266057] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.266321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.266402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.266487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.266570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.266650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.266729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.266815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.266894] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.266985] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.267096] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.268102] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.278798] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.279051] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.279903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.279979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.280049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.280121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.280196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.280269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.280340] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.280409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.280480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.280548] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.280617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.280688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.280758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.280839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.280907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.280977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.281046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.281114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.281183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.281259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.281336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.281412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.281490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.281567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.281641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.281719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.281804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.281883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.281958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.282033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.282108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.282192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.282274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.282354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.282432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.282509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.282581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.282655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.282728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.282807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.282881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.282954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.283025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.283103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.283176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.283251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.283326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.283403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.283477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.283548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.283623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.283697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.283769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.283850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.283924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.283995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.284067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.284143] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.284464] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.284468] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4152, lrc_seqno=4152, guc_id=0, flags=0x73 in no process [-1]
<7> [99.284470] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.284528] ------------[ cut here ]------------
<4> [99.284529] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.284530] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.284601] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.284659] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.284663] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.284666] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.284667] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.284668] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.284673] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.284739] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.284740] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.284742] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.284744] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.284745] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.284746] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.284747] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.284748] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.284749] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.284751] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.284752] PKRU: 55555554
<4> [99.284753] Call Trace:
<4> [99.284754] <TASK>
<4> [99.284759] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.284764] ? lock_release+0xce/0x280
<4> [99.284769] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.284774] process_one_work+0x22e/0x6b0
<4> [99.284780] worker_thread+0x1e8/0x3d0
<4> [99.284782] ? __pfx_worker_thread+0x10/0x10
<4> [99.284784] kthread+0x11f/0x250
<4> [99.284791] ? __pfx_kthread+0x10/0x10
<4> [99.284794] ret_from_fork+0x344/0x3a0
<4> [99.284796] ? __pfx_kthread+0x10/0x10
<4> [99.284799] ret_from_fork_asm+0x1a/0x30
<4> [99.284805] </TASK>
<4> [99.284806] irq event stamp: 543971
<4> [99.284808] hardirqs last enabled at (543977): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.284811] hardirqs last disabled at (543982): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.284813] softirqs last enabled at (542364): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.284815] softirqs last disabled at (542359): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.284817] ---[ end trace 0000000000000000 ]---
<7> [99.285457] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.286203] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4162, lrc_seqno=4162, guc_id=0, flags=0x73 in no process [-1]
<7> [99.286206] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.286270] ------------[ cut here ]------------
<4> [99.286271] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.286272] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.286340] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.286397] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.286402] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.286405] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.286406] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.286407] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.286411] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.286478] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.286480] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.286481] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.286483] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.286484] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.286485] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.286486] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.286487] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.286488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.286489] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.286491] PKRU: 55555554
<4> [99.286492] Call Trace:
<4> [99.286493] <TASK>
<4> [99.286497] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.286502] ? lock_release+0xce/0x280
<4> [99.286507] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.286512] process_one_work+0x22e/0x6b0
<4> [99.286518] worker_thread+0x1e8/0x3d0
<4> [99.286520] ? __pfx_worker_thread+0x10/0x10
<4> [99.286523] kthread+0x11f/0x250
<4> [99.286526] ? __pfx_kthread+0x10/0x10
<4> [99.286529] ret_from_fork+0x344/0x3a0
<4> [99.286531] ? __pfx_kthread+0x10/0x10
<4> [99.286534] ret_from_fork_asm+0x1a/0x30
<4> [99.286540] </TASK>
<4> [99.286541] irq event stamp: 544877
<4> [99.286542] hardirqs last enabled at (544883): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.286545] hardirqs last disabled at (544888): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.286547] softirqs last enabled at (542364): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.286549] softirqs last disabled at (542359): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.286551] ---[ end trace 0000000000000000 ]---
<6> [99.286553] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.286621] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.286628] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.286681] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.286686] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.286798] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.286900] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.287166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.287252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.287341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.287427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.287509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.287588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.287670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.287760] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.287873] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.287985] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.288994] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.298795] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.299053] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.299931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.300009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.300082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.300158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.300231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.300304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.300375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.300446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.300517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.300587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.300659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.300730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.300806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.300876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.300946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.301016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.301086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.301157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.301228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.301306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.301383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.301457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.301532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.301605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.301678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.301754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.301845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.301921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.301995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.302071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.302146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.302225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.302305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.302385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.302468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.302545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.302618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.302692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.302763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.302848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.302922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.302994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.303065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.303141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.303213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.303288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.303362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.303437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.303509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.303581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.303655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.303729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.303806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.303877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.303951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.304022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.304093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.304168] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.304474] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.304477] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4162, lrc_seqno=4162, guc_id=0, flags=0x73 in no process [-1]
<7> [99.304480] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.304538] ------------[ cut here ]------------
<4> [99.304539] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.304540] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.304610] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.304667] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.304672] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.304674] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.304675] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.304676] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.304681] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.304749] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.304751] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.304752] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.304754] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.304755] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.304756] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.304757] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.304758] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.304759] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.304761] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.304762] PKRU: 55555554
<4> [99.304763] Call Trace:
<4> [99.304764] <TASK>
<4> [99.304769] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.304774] ? lock_release+0xce/0x280
<4> [99.304779] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.304784] process_one_work+0x22e/0x6b0
<4> [99.304794] worker_thread+0x1e8/0x3d0
<4> [99.304797] ? __pfx_worker_thread+0x10/0x10
<4> [99.304799] kthread+0x11f/0x250
<4> [99.304802] ? __pfx_kthread+0x10/0x10
<4> [99.304806] ret_from_fork+0x344/0x3a0
<4> [99.304808] ? __pfx_kthread+0x10/0x10
<4> [99.304811] ret_from_fork_asm+0x1a/0x30
<4> [99.304817] </TASK>
<4> [99.304818] irq event stamp: 549359
<4> [99.304819] hardirqs last enabled at (549365): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.304822] hardirqs last disabled at (549370): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.304824] softirqs last enabled at (547846): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.304826] softirqs last disabled at (547841): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.304828] ---[ end trace 0000000000000000 ]---
<6> [99.304830] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.304896] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.305697] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.305801] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.305902] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.306005] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.306267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.306355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.306446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.306529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.306612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.306690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.306770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.306863] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.306955] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.307066] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.308074] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.318794] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.319051] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.319919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.319994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.320066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.320140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.320214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.320288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.320361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.320433] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.320506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.320578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.320651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.320723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.320805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.320876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.320947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.321018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.321087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.321157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.321226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.321305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.321383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.321457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.321534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.321606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.321679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.321755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.321844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.321920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.321994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.322068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.322147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.322229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.322308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.322387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.322463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.322538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.322611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.322684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.322758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.322845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.322921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.322993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.323065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.323139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.323211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.323286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.323363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.323439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.323512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.323585] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.323660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.323733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.323813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.323885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.323957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.324029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.324100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.324174] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.324490] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.324494] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4162, lrc_seqno=4162, guc_id=0, flags=0x73 in no process [-1]
<7> [99.324496] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.324554] ------------[ cut here ]------------
<4> [99.324555] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.324556] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.324626] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.324683] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.324687] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.324690] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.324691] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.324692] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.324696] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.324763] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.324765] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.324767] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.324768] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.324769] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.324770] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.324771] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.324773] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.324774] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.324775] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.324776] PKRU: 55555554
<4> [99.324777] Call Trace:
<4> [99.324778] <TASK>
<4> [99.324789] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.324793] ? lock_release+0xce/0x280
<4> [99.324799] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.324804] process_one_work+0x22e/0x6b0
<4> [99.324809] worker_thread+0x1e8/0x3d0
<4> [99.324812] ? __pfx_worker_thread+0x10/0x10
<4> [99.324814] kthread+0x11f/0x250
<4> [99.324817] ? __pfx_kthread+0x10/0x10
<4> [99.324821] ret_from_fork+0x344/0x3a0
<4> [99.324823] ? __pfx_kthread+0x10/0x10
<4> [99.324826] ret_from_fork_asm+0x1a/0x30
<4> [99.324832] </TASK>
<4> [99.324833] irq event stamp: 553873
<4> [99.324834] hardirqs last enabled at (553879): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.324837] hardirqs last disabled at (553884): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.324839] softirqs last enabled at (552360): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.324841] softirqs last disabled at (552351): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.324844] ---[ end trace 0000000000000000 ]---
<7> [99.325206] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.325211] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.325274] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.325421] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.325729] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4170, lrc_seqno=4170, guc_id=0, flags=0x73 in no process [-1]
<7> [99.325733] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.325808] ------------[ cut here ]------------
<4> [99.325809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.325810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.325877] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.325934] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.325938] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.325940] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.325941] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.325942] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.325947] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.326011] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.326013] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.326014] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.326016] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.326017] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.326018] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.326019] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.326020] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.326021] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.326022] CR2: 00007973fe76e000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.326023] PKRU: 55555554
<4> [99.326024] Call Trace:
<4> [99.326025] <TASK>
<4> [99.326030] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.326034] ? lock_release+0xce/0x280
<4> [99.326040] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.326044] process_one_work+0x22e/0x6b0
<4> [99.326050] worker_thread+0x1e8/0x3d0
<4> [99.326052] ? __pfx_worker_thread+0x10/0x10
<4> [99.326054] kthread+0x11f/0x250
<4> [99.326058] ? __pfx_kthread+0x10/0x10
<4> [99.326061] ret_from_fork+0x344/0x3a0
<4> [99.326063] ? __pfx_kthread+0x10/0x10
<4> [99.326065] ret_from_fork_asm+0x1a/0x30
<4> [99.326072] </TASK>
<4> [99.326073] irq event stamp: 554769
<4> [99.326074] hardirqs last enabled at (554775): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.326076] hardirqs last disabled at (554780): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.326078] softirqs last enabled at (552360): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.326080] softirqs last disabled at (552351): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.326082] ---[ end trace 0000000000000000 ]---
<6> [99.326084] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.326150] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.326157] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.326174] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.326273] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.326378] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.326749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.326899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.327032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.327155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.327280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.327402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.327524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.327650] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.327790] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.327951] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.328957] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.338793] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.339051] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.339918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.340000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.340075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.340148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.340221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.340294] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.340365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.340436] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.340506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.340577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.340647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.340716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.340795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.340865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.340935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.341006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.341077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.341150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.341222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.341302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.341380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.341454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.341531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.341605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.341679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.341756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.341843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.341922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.342000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.342075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.342149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.342229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.342307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.342385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.342461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.342535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.342606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.342678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.342750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.342835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.342909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.342985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.343061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.343137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.343211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.343284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.343355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.343428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.343498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.343570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.343644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.343719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.343803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.343878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.343950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.344021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.344092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.344169] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.344468] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.344472] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4170, lrc_seqno=4170, guc_id=0, flags=0x73 in no process [-1]
<7> [99.344475] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.344532] ------------[ cut here ]------------
<4> [99.344533] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.344534] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:4/204
<4> [99.344603] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.344660] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.344665] CPU: 10 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.344668] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.344669] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.344670] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.344675] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.344741] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.344743] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [99.344745] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.344746] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.344747] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.344748] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.344749] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.344750] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.344752] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.344753] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.344754] PKRU: 55555554
<4> [99.344755] Call Trace:
<4> [99.344756] <TASK>
<4> [99.344761] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.344766] ? lock_release+0xce/0x280
<4> [99.344772] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.344777] process_one_work+0x22e/0x6b0
<4> [99.344789] worker_thread+0x1e8/0x3d0
<4> [99.344791] ? __pfx_worker_thread+0x10/0x10
<4> [99.344794] kthread+0x11f/0x250
<4> [99.344797] ? __pfx_kthread+0x10/0x10
<4> [99.344801] ret_from_fork+0x344/0x3a0
<4> [99.344803] ? __pfx_kthread+0x10/0x10
<4> [99.344806] ret_from_fork_asm+0x1a/0x30
<4> [99.344812] </TASK>
<4> [99.344813] irq event stamp: 9803
<4> [99.344814] hardirqs last enabled at (9809): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.344817] hardirqs last disabled at (9814): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.344819] softirqs last enabled at (8490): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.344821] softirqs last disabled at (8441): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.344824] ---[ end trace 0000000000000000 ]---
<6> [99.344825] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.344893] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.345698] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.345839] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.345955] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.346206] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.346472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.346559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.346647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.346731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.346825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.346908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.346991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.347072] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.347163] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.347274] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.348280] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.358788] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.359044] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.359915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.359991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.360063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.360135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.360207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.360278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.360349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.360419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.360491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.360562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.360632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.360701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.360769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.360863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.360934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.361007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.361079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.361150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.361220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.361300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.361378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.361453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.361533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.361609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.361684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.361760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.361847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.361923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.361996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.362073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.362152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.362235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.362316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.362396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.362473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.362549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.362624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.362697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.362768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.362854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.362927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.363000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.363074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.363150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.363223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.363298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.363372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.363447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.363519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.363591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.363667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.363740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.363821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.363892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.363966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.364040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.364117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.364195] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.364493] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.364497] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4170, lrc_seqno=4170, guc_id=0, flags=0x73 in no process [-1]
<7> [99.364499] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.364557] ------------[ cut here ]------------
<4> [99.364558] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.364559] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:4/204
<4> [99.364629] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.364686] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.364691] CPU: 8 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.364693] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.364694] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.364696] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.364700] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.364766] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.364768] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [99.364770] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.364771] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.364772] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.364773] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.364774] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.364775] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.364777] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.364778] CR2: 00007973fe76e000 CR3: 000000011d8e7006 CR4: 0000000000f72ef0
<4> [99.364779] PKRU: 55555554
<4> [99.364785] Call Trace:
<4> [99.364786] <TASK>
<4> [99.364792] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.364797] ? lock_release+0xce/0x280
<4> [99.364802] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.364807] process_one_work+0x22e/0x6b0
<4> [99.364813] worker_thread+0x1e8/0x3d0
<4> [99.364815] ? __pfx_worker_thread+0x10/0x10
<4> [99.364818] kthread+0x11f/0x250
<4> [99.364821] ? __pfx_kthread+0x10/0x10
<4> [99.364824] ret_from_fork+0x344/0x3a0
<4> [99.364826] ? __pfx_kthread+0x10/0x10
<4> [99.364829] ret_from_fork_asm+0x1a/0x30
<4> [99.364836] </TASK>
<4> [99.364837] irq event stamp: 14369
<4> [99.364838] hardirqs last enabled at (14375): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.364840] hardirqs last disabled at (14380): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.364843] softirqs last enabled at (12856): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.364845] softirqs last disabled at (12849): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.364848] ---[ end trace 0000000000000000 ]---
<7> [99.365556] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.365645] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.365813] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4180, lrc_seqno=4180, guc_id=0, flags=0x73 in no process [-1]
<7> [99.365819] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.365927] ------------[ cut here ]------------
<4> [99.365929] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.365931] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#12: kworker/u64:11/211
<7> [99.365954] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.366046] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep
<7> [99.366056] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.366104] intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.366139] CPU: 12 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.366142] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.366144] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.366145] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.366153] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.366265] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.366267] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.366270] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.366272] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.366273] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.366275] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.366277] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.366279] FS: 0000000000000000(0000) GS:ffff8888db2da000(0000) knlGS:0000000000000000
<4> [99.366281] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.366282] CR2: 0000797c95542278 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.366284] PKRU: 55555554
<4> [99.366286] Call Trace:
<4> [99.366287] <TASK>
<4> [99.366296] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.366303] ? lock_release+0xce/0x280
<4> [99.366312] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.366320] process_one_work+0x22e/0x6b0
<4> [99.366330] worker_thread+0x1e8/0x3d0
<4> [99.366334] ? __pfx_worker_thread+0x10/0x10
<4> [99.366337] kthread+0x11f/0x250
<4> [99.366341] ? __pfx_kthread+0x10/0x10
<4> [99.366346] ret_from_fork+0x344/0x3a0
<4> [99.366349] ? __pfx_kthread+0x10/0x10
<4> [99.366352] ret_from_fork_asm+0x1a/0x30
<4> [99.366363] </TASK>
<4> [99.366365] irq event stamp: 104805
<4> [99.366366] hardirqs last enabled at (104811): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.366370] hardirqs last disabled at (104816): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.366372] softirqs last enabled at (102952): [<ffffffff813346d3>] kernel_fpu_end+0x53/0x70
<4> [99.366375] softirqs last disabled at (102950): [<ffffffff81334db4>] kernel_fpu_begin_mask+0xc4/0x120
<4> [99.366378] ---[ end trace 0000000000000000 ]---
<6> [99.366381] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.366491] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.366500] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.366550] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.366890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.367023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.367157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.367286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.367414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.367540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.367668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.367800] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.367940] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.368100] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.369461] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.379790] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.380105] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.381094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.381210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.381325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.381441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.381557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.381673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.381803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.381921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.382037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.382153] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.382269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.382384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.382500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.382616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.382731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.382853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.382970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.383086] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.383201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.383322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.383443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.383562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.383683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.383806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.383926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.384048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.384174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.384296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.384417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.384538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.384660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.384793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.384921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.385047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.385172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.385295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.385415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.385536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.385655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.385778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.385904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.386026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.386145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.386269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.386389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.386510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.386628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.386749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.386872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.386992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.387114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.387234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.387355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.387474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.387594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.387713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.387835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.387960] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.388315] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.388322] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4180, lrc_seqno=4180, guc_id=0, flags=0x73 in no process [-1]
<7> [99.388325] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.388422] ------------[ cut here ]------------
<4> [99.388424] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.388425] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#12: kworker/u64:11/211
<4> [99.388534] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.388615] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.388622] CPU: 12 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.388625] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.388626] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.388628] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.388635] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.388742] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.388744] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.388747] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.388748] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.388750] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.388752] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.388753] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.388755] FS: 0000000000000000(0000) GS:ffff8888db2da000(0000) knlGS:0000000000000000
<4> [99.388757] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.388758] CR2: 0000797c95542278 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.388760] PKRU: 55555554
<4> [99.388761] Call Trace:
<4> [99.388763] <TASK>
<4> [99.388771] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.388785] ? lock_release+0xce/0x280
<4> [99.388793] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.388802] process_one_work+0x22e/0x6b0
<4> [99.388811] worker_thread+0x1e8/0x3d0
<4> [99.388815] ? __pfx_worker_thread+0x10/0x10
<4> [99.388818] kthread+0x11f/0x250
<4> [99.388822] ? __pfx_kthread+0x10/0x10
<4> [99.388827] ret_from_fork+0x344/0x3a0
<4> [99.388830] ? __pfx_kthread+0x10/0x10
<4> [99.388833] ret_from_fork_asm+0x1a/0x30
<4> [99.388844] </TASK>
<4> [99.388846] irq event stamp: 109343
<4> [99.388847] hardirqs last enabled at (109349): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.388850] hardirqs last disabled at (109354): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.388853] softirqs last enabled at (107874): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.388856] softirqs last disabled at (107869): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.388859] ---[ end trace 0000000000000000 ]---
<6> [99.388861] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.388972] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.388989] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.390282] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.390431] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.390580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.390855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.390947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.391040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.391124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.391207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.391289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.391370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.391450] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.391541] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.391653] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.392662] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.402788] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.403044] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.403917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.403993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.404066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.404138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.404210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.404282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.404352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.404423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.404493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.404565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.404638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.404710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.404792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.404864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.404934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.405006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.405076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.405146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.405217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.405295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.405373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.405450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.405528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.405602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.405676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.405751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.405844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.405921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.405999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.406075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.406151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.406232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.406312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.406391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.406468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.406543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.406615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.406687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.406759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.406847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.406922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.406994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.407066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.407145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.407220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.407293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.407366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.407438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.407509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.407579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.407653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.407727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.407806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.407879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.407953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.408028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.408101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.408177] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.408478] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.408482] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4180, lrc_seqno=4180, guc_id=0, flags=0x73 in no process [-1]
<7> [99.408485] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.408543] ------------[ cut here ]------------
<4> [99.408544] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.408545] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:11/211
<4> [99.408614] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.408672] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.408677] CPU: 7 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.408680] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.408681] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.408682] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.408687] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.408754] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.408756] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.408758] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.408759] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.408760] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.408761] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.408762] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.408763] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.408765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.408766] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.408767] PKRU: 55555554
<4> [99.408768] Call Trace:
<4> [99.408769] <TASK>
<4> [99.408775] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.408784] ? lock_release+0xce/0x280
<4> [99.408790] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.408795] process_one_work+0x22e/0x6b0
<4> [99.408801] worker_thread+0x1e8/0x3d0
<4> [99.408803] ? __pfx_worker_thread+0x10/0x10
<4> [99.408806] kthread+0x11f/0x250
<4> [99.408809] ? __pfx_kthread+0x10/0x10
<4> [99.408812] ret_from_fork+0x344/0x3a0
<4> [99.408814] ? __pfx_kthread+0x10/0x10
<4> [99.408817] ret_from_fork_asm+0x1a/0x30
<4> [99.408824] </TASK>
<4> [99.408825] irq event stamp: 113825
<4> [99.408826] hardirqs last enabled at (113831): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.408829] hardirqs last disabled at (113836): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.408831] softirqs last enabled at (112420): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.408833] softirqs last disabled at (112409): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.408836] ---[ end trace 0000000000000000 ]---
<7> [99.409259] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.409267] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.409281] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.409705] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.409712] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4188, lrc_seqno=4188, guc_id=0, flags=0x73 in no process [-1]
<7> [99.409715] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.409790] ------------[ cut here ]------------
<4> [99.409792] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.409795] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:11/211
<4> [99.409884] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.409951] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.409956] CPU: 7 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.409959] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.409960] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.409962] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.409967] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.410041] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.410042] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.410044] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.410045] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.410046] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.410047] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.410049] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.410050] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.410051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.410052] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.410053] PKRU: 55555554
<4> [99.410054] Call Trace:
<4> [99.410055] <TASK>
<4> [99.410060] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.410064] ? lock_release+0xce/0x280
<4> [99.410070] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.410075] process_one_work+0x22e/0x6b0
<4> [99.410080] worker_thread+0x1e8/0x3d0
<4> [99.410083] ? __pfx_worker_thread+0x10/0x10
<4> [99.410085] kthread+0x11f/0x250
<4> [99.410088] ? __pfx_kthread+0x10/0x10
<4> [99.410092] ret_from_fork+0x344/0x3a0
<4> [99.410094] ? __pfx_kthread+0x10/0x10
<4> [99.410096] ret_from_fork_asm+0x1a/0x30
<4> [99.410103] </TASK>
<4> [99.410104] irq event stamp: 114727
<4> [99.410105] hardirqs last enabled at (114733): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.410107] hardirqs last disabled at (114738): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.410109] softirqs last enabled at (113960): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.410112] softirqs last disabled at (113955): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.410114] ---[ end trace 0000000000000000 ]---
<6> [99.410116] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.410183] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.410192] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.410219] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.410318] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.410422] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.410770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.410911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.411048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.411177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.411305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.411432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.411561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.411687] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.411830] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.411993] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.413341] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.423788] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.424103] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.425093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.425209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.425324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.425441] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.425557] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.425673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.425795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.425913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.426029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.426145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.426260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.426376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.426492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.426608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.426724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.426847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.426965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.427081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.427197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.427317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.427438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.427558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.427679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.427813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.427934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.428056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.428181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.428304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.428426] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.428548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.428670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.428803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.428931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.429057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.429182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.429306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.429427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.429548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.429667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.429794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.429918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.430039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.430158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.430283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.430403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.430524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.430642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.430763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.430886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.431006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.431129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.431249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.431370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.431489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.431610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.431729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.431852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.431976] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.432328] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.432333] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4188, lrc_seqno=4188, guc_id=0, flags=0x73 in no process [-1]
<7> [99.432335] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.432396] ------------[ cut here ]------------
<4> [99.432397] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.432398] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:0/12
<4> [99.432471] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.432529] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.432534] CPU: 10 UID: 0 PID: 12 Comm: kworker/u64:0 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.432537] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.432538] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.432539] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.432544] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.432614] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.432616] RSP: 0018:ffffc900000b7c98 EFLAGS: 00010246
<4> [99.432618] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.432619] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.432620] RBP: ffffc900000b7da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.432621] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.432622] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.432624] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.432625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.432626] CR2: 00007973fe770000 CR3: 0000000194f43002 CR4: 0000000000f72ef0
<4> [99.432628] PKRU: 55555554
<4> [99.432629] Call Trace:
<4> [99.432629] <TASK>
<4> [99.432635] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.432639] ? lock_release+0xce/0x280
<4> [99.432645] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.432650] process_one_work+0x22e/0x6b0
<4> [99.432655] worker_thread+0x1e8/0x3d0
<4> [99.432658] ? __pfx_worker_thread+0x10/0x10
<4> [99.432660] kthread+0x11f/0x250
<4> [99.432663] ? __pfx_kthread+0x10/0x10
<4> [99.432667] ret_from_fork+0x344/0x3a0
<4> [99.432669] ? __pfx_kthread+0x10/0x10
<4> [99.432672] ret_from_fork_asm+0x1a/0x30
<4> [99.432678] </TASK>
<4> [99.432679] irq event stamp: 38977
<4> [99.432680] hardirqs last enabled at (38983): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.432682] hardirqs last disabled at (38988): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.432684] softirqs last enabled at (37502): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.432687] softirqs last disabled at (37495): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.432689] ---[ end trace 0000000000000000 ]---
<6> [99.432691] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.432759] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.432766] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.433680] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.433807] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.433922] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.434194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.434285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.434376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.434461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.434547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.434628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.434713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.434801] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.434894] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.435005] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.436014] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.446734] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.447003] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.447882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.447957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.448030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.448103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.448177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.448247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.448319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.448389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.448459] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.448531] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.448603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.448673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.448743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.448823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.448893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.448963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.449031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.449099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.449169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.449246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.449322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.449397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.449472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.449544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.449618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.449695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.449773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.449864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.449940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.450014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.450087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.450167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.450247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.450329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.450407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.450481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.450553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.450626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.450700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.450787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.450863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.450935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.451007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.451082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.451154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.451227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.451300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.451375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.451446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.451519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.451595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.451668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.451742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.451825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.451900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.451972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.452043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.452117] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.452420] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.453224] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4188, lrc_seqno=4188, guc_id=0, flags=0x73 in no process [-1]
<7> [99.453226] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.453286] ------------[ cut here ]------------
<4> [99.453287] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.453288] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.453356] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.453413] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.453418] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.453421] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.453422] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.453423] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.453428] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.453494] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.453496] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.453498] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.453499] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.453500] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.453501] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.453502] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.453503] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.453505] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.453506] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.453507] PKRU: 55555554
<4> [99.453508] Call Trace:
<4> [99.453509] <TASK>
<4> [99.453515] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.453519] ? lock_release+0xce/0x280
<4> [99.453525] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.453530] process_one_work+0x22e/0x6b0
<4> [99.453536] worker_thread+0x1e8/0x3d0
<4> [99.453538] ? __pfx_worker_thread+0x10/0x10
<4> [99.453541] kthread+0x11f/0x250
<4> [99.453544] ? __pfx_kthread+0x10/0x10
<4> [99.453547] ret_from_fork+0x344/0x3a0
<4> [99.453549] ? __pfx_kthread+0x10/0x10
<4> [99.453552] ret_from_fork_asm+0x1a/0x30
<4> [99.453558] </TASK>
<4> [99.453559] irq event stamp: 1594115
<4> [99.453560] hardirqs last enabled at (1594121): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.453563] hardirqs last disabled at (1594126): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.453565] softirqs last enabled at (1592710): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.453568] softirqs last disabled at (1592701): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.453570] ---[ end trace 0000000000000000 ]---
<7> [99.453713] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.453847] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [99.453999] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4198, lrc_seqno=4198, guc_id=0, flags=0x73 in no process [-1]
<7> [99.454002] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.454086] ------------[ cut here ]------------
<4> [99.454087] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [99.454088] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.454089] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:11/211
<4> [99.454178] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.454246] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.454251] CPU: 7 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.454255] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.454256] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.454257] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.454263] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.454348] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.454350] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.454352] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.454353] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.454354] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.454356] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.454356] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.454358] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.454359] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.454360] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.454362] PKRU: 55555554
<4> [99.454364] Call Trace:
<4> [99.454366] <TASK>
<4> [99.454375] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.454380] ? lock_release+0xce/0x280
<4> [99.454385] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.454390] process_one_work+0x22e/0x6b0
<4> [99.454396] worker_thread+0x1e8/0x3d0
<4> [99.454399] ? __pfx_worker_thread+0x10/0x10
<4> [99.454401] kthread+0x11f/0x250
<4> [99.454404] ? __pfx_kthread+0x10/0x10
<4> [99.454407] ret_from_fork+0x344/0x3a0
<4> [99.454409] ? __pfx_kthread+0x10/0x10
<4> [99.454414] ret_from_fork_asm+0x1a/0x30
<4> [99.454425] </TASK>
<4> [99.454427] irq event stamp: 116635
<4> [99.454428] hardirqs last enabled at (116641): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.454433] hardirqs last disabled at (116646): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.454436] softirqs last enabled at (115010): [<ffffffff813346d3>] kernel_fpu_end+0x53/0x70
<4> [99.454439] softirqs last disabled at (115008): [<ffffffff81334db4>] kernel_fpu_begin_mask+0xc4/0x120
<4> [99.454443] ---[ end trace 0000000000000000 ]---
<6> [99.454445] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.454524] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.454530] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.454580] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.454848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.454935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.455024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.455108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.455193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.455274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.455355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.455433] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.455525] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.455637] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.456646] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.466783] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.467039] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.467920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.467995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.468066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.468139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.468211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.468282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.468353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.468424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.468494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.468563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.468634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.468705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.468792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.468863] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.468933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.469002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.469071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.469141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.469210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.469288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.469367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.469444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.469519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.469591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.469665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.469740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.469833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.469911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.469986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.470061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.470139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.470220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.470301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.470380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.470457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.470533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.470607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.470680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.470752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.470835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.470913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.470988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.471062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.471139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.471213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.471287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.471358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.471431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.471503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.471575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.471649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.471725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.471806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.471878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.471953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.472024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.472096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.472171] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.472472] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.472476] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4198, lrc_seqno=4198, guc_id=0, flags=0x73 in no process [-1]
<7> [99.472478] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.472535] ------------[ cut here ]------------
<4> [99.472536] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.472537] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:11/211
<4> [99.472606] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.472663] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.472668] CPU: 7 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.472670] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.472671] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.472672] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.472677] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.472743] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.472745] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.472747] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.472748] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.472749] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.472750] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.472751] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.472753] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.472754] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.472755] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.472756] PKRU: 55555554
<4> [99.472757] Call Trace:
<4> [99.472758] <TASK>
<4> [99.472763] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.472768] ? lock_release+0xce/0x280
<4> [99.472777] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.472782] process_one_work+0x22e/0x6b0
<4> [99.472788] worker_thread+0x1e8/0x3d0
<4> [99.472791] ? __pfx_worker_thread+0x10/0x10
<4> [99.472793] kthread+0x11f/0x250
<4> [99.472797] ? __pfx_kthread+0x10/0x10
<4> [99.472800] ret_from_fork+0x344/0x3a0
<4> [99.472802] ? __pfx_kthread+0x10/0x10
<4> [99.472805] ret_from_fork_asm+0x1a/0x30
<4> [99.472812] </TASK>
<4> [99.472813] irq event stamp: 121163
<4> [99.472814] hardirqs last enabled at (121169): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.472816] hardirqs last disabled at (121174): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.472818] softirqs last enabled at (119650): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.472821] softirqs last disabled at (119643): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.472823] ---[ end trace 0000000000000000 ]---
<6> [99.472824] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.472890] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.473692] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.473795] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.473911] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.474013] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.474276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.474359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.474447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.474530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.474613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.474695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.474782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.474862] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.474953] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.475063] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.476070] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.486784] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.487073] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.487981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.488056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.488129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.488202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.488275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.488347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.488420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.488490] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.488562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.488635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.488707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.488780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.488851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.488923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.488995] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.489066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.489139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.489210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.489279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.489356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.489433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.489508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.489584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.489658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.489733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.489820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.489899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.489976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.490049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.490124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.490198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.490277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.490355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.490434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.490510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.490584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.490656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.490728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.490807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.490882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.490954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.491027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.491097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.491173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.491244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.491319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.491393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.491468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.491540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.491612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.491687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.491760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.491848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.491923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.491997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.492070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.492143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.492218] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.492528] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.492532] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4198, lrc_seqno=4198, guc_id=0, flags=0x73 in no process [-1]
<7> [99.492534] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.492592] ------------[ cut here ]------------
<4> [99.492593] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.492594] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:11/211
<4> [99.492664] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.492721] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.492725] CPU: 7 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.492728] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.492729] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.492730] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.492735] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.492811] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.492813] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.492815] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.492817] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.492818] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.492819] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.492820] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.492822] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.492823] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.492824] CR2: 00007412107946f4 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.492826] PKRU: 55555554
<4> [99.492827] Call Trace:
<4> [99.492828] <TASK>
<4> [99.492833] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.492838] ? lock_release+0xce/0x280
<4> [99.492843] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.492848] process_one_work+0x22e/0x6b0
<4> [99.492854] worker_thread+0x1e8/0x3d0
<4> [99.492857] ? __pfx_worker_thread+0x10/0x10
<4> [99.492859] kthread+0x11f/0x250
<4> [99.492862] ? __pfx_kthread+0x10/0x10
<4> [99.492865] ret_from_fork+0x344/0x3a0
<4> [99.492868] ? __pfx_kthread+0x10/0x10
<4> [99.492870] ret_from_fork_asm+0x1a/0x30
<4> [99.492877] </TASK>
<4> [99.492878] irq event stamp: 125661
<4> [99.492878] hardirqs last enabled at (125667): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.492881] hardirqs last disabled at (125672): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.492884] softirqs last enabled at (124348): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.492886] softirqs last disabled at (124331): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.492888] ---[ end trace 0000000000000000 ]---
<5> [99.493844] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4204, lrc_seqno=4204, guc_id=0, flags=0x73 in no process [-1]
<7> [99.493848] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.493849] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.493909] ------------[ cut here ]------------
<4> [99.493910] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.493911] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.493983] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit
<7> [99.493953] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.494000] intel_rapl_msr
<4> [99.494001] intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.494048] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.494051] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.494052] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.494053] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.494058] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.494127] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.494129] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.494131] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.494132] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.494133] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.494134] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.494135] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.494137] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.494138] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.494139] CR2: 00007973fe772000 CR3: 0000000194f43005 CR4: 0000000000f72ef0
<4> [99.494141] PKRU: 55555554
<4> [99.494141] Call Trace:
<4> [99.494142] <TASK>
<4> [99.494147] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.494152] ? lock_release+0xce/0x280
<4> [99.494157] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.494162] process_one_work+0x22e/0x6b0
<4> [99.494168] worker_thread+0x1e8/0x3d0
<4> [99.494171] ? __pfx_worker_thread+0x10/0x10
<4> [99.494173] kthread+0x11f/0x250
<4> [99.494176] ? __pfx_kthread+0x10/0x10
<4> [99.494180] ret_from_fork+0x344/0x3a0
<4> [99.494182] ? __pfx_kthread+0x10/0x10
<4> [99.494185] ret_from_fork_asm+0x1a/0x30
<4> [99.494191] </TASK>
<4> [99.494193] irq event stamp: 558007
<4> [99.494194] hardirqs last enabled at (558013): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.494196] hardirqs last disabled at (558018): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.494199] softirqs last enabled at (556364): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.494201] softirqs last disabled at (556343): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.494203] ---[ end trace 0000000000000000 ]---
<6> [99.494205] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.494276] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.494282] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.494329] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.494594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.494679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.494767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.494865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.494946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.495026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.495129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.495208] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.495208] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.495299] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.495412] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.496418] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.506779] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.507034] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.507879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.507955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.508027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.508117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.508190] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.508260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.508331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.508401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.508473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.508544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.508613] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.508684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.508754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.508835] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.508903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.508972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.509041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.509110] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.509181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.509257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.509334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.509410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.509488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.509562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.509636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.509711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.509791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.509868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.509942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.510016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.510091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.510171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.510250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.510329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.510405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.510480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.510552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.510624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.510696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.510775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.510848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.510922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.510994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.511071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.511144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.511218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.511292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.511370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.511443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.511516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.511592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.511665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.511738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.511822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.511894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.511964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.512036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.512111] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.512430] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.512434] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4204, lrc_seqno=4204, guc_id=0, flags=0x73 in no process [-1]
<7> [99.512436] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.512494] ------------[ cut here ]------------
<4> [99.512495] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.512496] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.512565] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.512622] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.512627] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.512629] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.512630] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.512631] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.512636] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.512703] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.512705] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.512707] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.512708] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.512709] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.512710] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.512711] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.512712] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.512714] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.512715] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.512716] PKRU: 55555554
<4> [99.512717] Call Trace:
<4> [99.512718] <TASK>
<4> [99.512723] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.512728] ? lock_release+0xce/0x280
<4> [99.512733] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.512738] process_one_work+0x22e/0x6b0
<4> [99.512743] worker_thread+0x1e8/0x3d0
<4> [99.512746] ? __pfx_worker_thread+0x10/0x10
<4> [99.512748] kthread+0x11f/0x250
<4> [99.512751] ? __pfx_kthread+0x10/0x10
<4> [99.512754] ret_from_fork+0x344/0x3a0
<4> [99.512756] ? __pfx_kthread+0x10/0x10
<4> [99.512759] ret_from_fork_asm+0x1a/0x30
<4> [99.512765] </TASK>
<4> [99.512766] irq event stamp: 562477
<4> [99.512767] hardirqs last enabled at (562483): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.512770] hardirqs last disabled at (562488): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.512775] softirqs last enabled at (561172): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.512778] softirqs last disabled at (561167): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.512780] ---[ end trace 0000000000000000 ]---
<6> [99.512782] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.512848] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.513655] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.513741] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.513850] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.513954] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.514219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.514302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.514392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.514478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.514560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.514640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.514721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.514805] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.514896] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.515007] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.516013] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.525780] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.526035] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.526869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.526945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.527016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.527089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.527160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.527233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.527303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.527373] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.527443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.527514] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.527584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.527655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.527725] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.527802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.527871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.527941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.528010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.528080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.528148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.528225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.528300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.528376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.528453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.528527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.528599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.528674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.528754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.528844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.528920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.528995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.529071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.529151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.529231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.529311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.529387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.529463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.529536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.529609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.529681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.529756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.529840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.529916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.529990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.530068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.530141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.530214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.530285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.530360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.530432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.530503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.530579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.530652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.530725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.530801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.530874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.530947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.531018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.531092] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.531443] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.531447] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4204, lrc_seqno=4204, guc_id=0, flags=0x73 in no process [-1]
<7> [99.531449] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.531506] ------------[ cut here ]------------
<4> [99.531507] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.531508] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.531581] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.531637] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.531642] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.531644] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.531645] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.531646] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.531651] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.531719] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.531720] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.531722] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.531723] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.531725] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.531726] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.531727] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.531728] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.531729] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.531730] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.531732] PKRU: 55555554
<4> [99.531733] Call Trace:
<4> [99.531733] <TASK>
<4> [99.531738] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.531743] ? lock_release+0xce/0x280
<4> [99.531748] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.531753] process_one_work+0x22e/0x6b0
<4> [99.531759] worker_thread+0x1e8/0x3d0
<4> [99.531761] ? __pfx_worker_thread+0x10/0x10
<4> [99.531764] kthread+0x11f/0x250
<4> [99.531767] ? __pfx_kthread+0x10/0x10
<4> [99.531776] ret_from_fork+0x344/0x3a0
<4> [99.531779] ? __pfx_kthread+0x10/0x10
<4> [99.531783] ret_from_fork_asm+0x1a/0x30
<4> [99.531789] </TASK>
<4> [99.531790] irq event stamp: 566965
<4> [99.531791] hardirqs last enabled at (566971): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.531794] hardirqs last disabled at (566976): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.531796] softirqs last enabled at (565460): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.531799] softirqs last disabled at (565451): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.531801] ---[ end trace 0000000000000000 ]---
<5> [99.532620] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4207, lrc_seqno=4207, guc_id=0, flags=0x73 in no process [-1]
<7> [99.532623] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.532695] ------------[ cut here ]------------
<4> [99.532696] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.532697] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.532783] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.532849] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.532853] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.532856] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.532856] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.532858] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.532862] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.532932] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.532934] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.532936] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.532937] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.532938] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.532939] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.532940] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<7> [99.532938] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.532941] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.532943] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.532944] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.532946] PKRU: 55555554
<4> [99.532947] Call Trace:
<4> [99.532948] <TASK>
<4> [99.532952] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.532957] ? lock_release+0xce/0x280
<4> [99.532962] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.532967] process_one_work+0x22e/0x6b0
<4> [99.532973] worker_thread+0x1e8/0x3d0
<4> [99.532975] ? __pfx_worker_thread+0x10/0x10
<4> [99.532977] kthread+0x11f/0x250
<4> [99.532981] ? __pfx_kthread+0x10/0x10
<4> [99.532984] ret_from_fork+0x344/0x3a0
<4> [99.532986] ? __pfx_kthread+0x10/0x10
<4> [99.532989] ret_from_fork_asm+0x1a/0x30
<4> [99.532995] </TASK>
<4> [99.532996] irq event stamp: 567833
<4> [99.532997] hardirqs last enabled at (567839): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.532999] hardirqs last disabled at (567844): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.533001] softirqs last enabled at (565460): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.533004] softirqs last disabled at (565451): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.533006] ---[ end trace 0000000000000000 ]---
<6> [99.533007] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.533074] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.533080] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.533132] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.533212] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.533221] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.533230] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.533332] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.533333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.533599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.533685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.533778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.533861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.533944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.534025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.534104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.534184] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.534279] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.534391] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.535398] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.545776] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.546033] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.546865] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.546942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.547014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.547087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.547158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.547231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.547302] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.547374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.547446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.547519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.547592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.547665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.547735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.547816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.547887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.547958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.548026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.548094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.548164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.548242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.548318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.548392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.548469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.548543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.548616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.548693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.548777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.548854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.548930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.549005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.549080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.549159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.549239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.549318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.549395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.549471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.549544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.549616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.549687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.549762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.549849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.549922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.549993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.550069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.550141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.550214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.550288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.550363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.550434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.550506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.550580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.550653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.550724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.550802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.550876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.550948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.551019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.551092] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.551507] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.552309] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4207, lrc_seqno=4207, guc_id=0, flags=0x73 in no process [-1]
<7> [99.552312] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.552372] ------------[ cut here ]------------
<4> [99.552373] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.552374] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.552443] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.552500] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.552505] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.552507] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.552508] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.552509] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.552514] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.552581] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.552583] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.552585] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.552586] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.552587] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.552588] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.552589] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.552590] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.552592] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.552593] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.552594] PKRU: 55555554
<4> [99.552595] Call Trace:
<4> [99.552596] <TASK>
<4> [99.552601] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.552606] ? lock_release+0xce/0x280
<4> [99.552611] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.552616] process_one_work+0x22e/0x6b0
<4> [99.552622] worker_thread+0x1e8/0x3d0
<4> [99.552624] ? __pfx_worker_thread+0x10/0x10
<4> [99.552627] kthread+0x11f/0x250
<4> [99.552630] ? __pfx_kthread+0x10/0x10
<4> [99.552633] ret_from_fork+0x344/0x3a0
<4> [99.552635] ? __pfx_kthread+0x10/0x10
<4> [99.552638] ret_from_fork_asm+0x1a/0x30
<4> [99.552644] </TASK>
<4> [99.552645] irq event stamp: 572335
<4> [99.552646] hardirqs last enabled at (572341): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.552649] hardirqs last disabled at (572346): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.552651] softirqs last enabled at (570828): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.552653] softirqs last disabled at (570823): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.552655] ---[ end trace 0000000000000000 ]---
<6> [99.552657] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.552725] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.552731] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.552743] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.552853] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.552953] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.553225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.553313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.553405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.553489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.553571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.553651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.553732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.553823] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.553913] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.554026] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.555034] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.564776] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.565032] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.565900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.565976] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.566048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.566121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.566192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.566265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.566336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.566409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.566480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.566550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.566621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.566690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.566762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.566846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.566917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.566986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.567056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.567125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.567193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.567269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.567347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.567422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.567497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.567575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.567648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.567726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.567813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.567890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.567964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.568038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.568112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.568191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.568269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.568348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.568425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.568500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.568572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.568645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.568720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.568801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.568876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.568947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.569018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.569093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.569166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.569239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.569311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.569386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.569461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.569531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.569605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.569678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.569752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.569835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.569910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.569981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.570051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.570128] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.570441] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.570445] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4207, lrc_seqno=4207, guc_id=0, flags=0x73 in no process [-1]
<7> [99.570448] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.570506] ------------[ cut here ]------------
<4> [99.570507] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.570508] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.570581] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.570639] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.570644] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.570647] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.570648] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.570650] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.570655] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.570721] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.570723] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.570725] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.570726] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.570727] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.570728] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.570729] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.570731] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.570732] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.570733] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.570735] PKRU: 55555554
<4> [99.570736] Call Trace:
<4> [99.570737] <TASK>
<4> [99.570742] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.570747] ? lock_release+0xce/0x280
<4> [99.570752] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.570758] process_one_work+0x22e/0x6b0
<4> [99.570763] worker_thread+0x1e8/0x3d0
<4> [99.570769] ? __pfx_worker_thread+0x10/0x10
<4> [99.570771] kthread+0x11f/0x250
<4> [99.570775] ? __pfx_kthread+0x10/0x10
<4> [99.570778] ret_from_fork+0x344/0x3a0
<4> [99.570780] ? __pfx_kthread+0x10/0x10
<4> [99.570783] ret_from_fork_asm+0x1a/0x30
<4> [99.570789] </TASK>
<4> [99.570790] irq event stamp: 192289
<4> [99.570792] hardirqs last enabled at (192295): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.570794] hardirqs last disabled at (192300): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.570796] softirqs last enabled at (190776): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.570799] softirqs last disabled at (190771): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.570801] ---[ end trace 0000000000000000 ]---
<5> [99.572397] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4217, lrc_seqno=4217, guc_id=0, flags=0x73 in no process [-1]
<7> [99.572401] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.572499] ------------[ cut here ]------------
<4> [99.572500] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.572501] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.572572] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.572630] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.572634] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.572637] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.572638] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.572639] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.572644] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.572711] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.572712] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.572715] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.572716] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.572717] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.572718] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.572719] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.572720] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.572722] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.572723] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.572724] PKRU: 55555554
<4> [99.572725] Call Trace:
<4> [99.572726] <TASK>
<4> [99.572731] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.572736] ? lock_release+0xce/0x280
<4> [99.572742] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.572747] process_one_work+0x22e/0x6b0
<4> [99.572753] worker_thread+0x1e8/0x3d0
<4> [99.572755] ? __pfx_worker_thread+0x10/0x10
<4> [99.572757] kthread+0x11f/0x250
<4> [99.572760] ? __pfx_kthread+0x10/0x10
<4> [99.572764] ret_from_fork+0x344/0x3a0
<4> [99.572766] ? __pfx_kthread+0x10/0x10
<4> [99.572774] ret_from_fork_asm+0x1a/0x30
<4> [99.572781] </TASK>
<4> [99.572782] irq event stamp: 193209
<4> [99.572783] hardirqs last enabled at (193215): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.572786] hardirqs last disabled at (193220): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.572788] softirqs last enabled at (193192): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.572791] softirqs last disabled at (193185): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.572793] ---[ end trace 0000000000000000 ]---
<6> [99.572794] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.572863] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.572868] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.572921] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.573019] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.573121] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.573386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.573471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.573558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.573645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.573730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.573825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.573907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.573985] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.574076] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.574189] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.575199] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.585776] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.586031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.586902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.586978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.587050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.587122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.587196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.587268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.587339] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.587410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.587481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.587552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.587623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.587695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.587770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.587842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.587911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.587981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.588049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.588119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.588189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.588267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.588343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.588417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.588494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.588568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.588641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.588716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.588807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.588883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.588956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.589032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.589107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.589186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.589267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.589345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.589421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.589497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.589570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.589642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.589714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.589801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.589878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.589954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.590029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.590105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.590178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.590251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.590323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.590397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.590469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.590545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.590622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.590696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.590773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.590846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.590920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.590991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.591063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.591138] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.591426] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.591429] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4217, lrc_seqno=4217, guc_id=0, flags=0x73 in no process [-1]
<7> [99.591432] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.591490] ------------[ cut here ]------------
<4> [99.591491] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.591492] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.591560] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.591617] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.591622] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.591625] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.591626] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.591627] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.591631] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.591697] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.591699] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.591701] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.591702] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.591703] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.591704] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.591705] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.591706] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.591708] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.591709] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.591710] PKRU: 55555554
<4> [99.591711] Call Trace:
<4> [99.591712] <TASK>
<4> [99.591717] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.591722] ? lock_release+0xce/0x280
<4> [99.591727] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.591732] process_one_work+0x22e/0x6b0
<4> [99.591738] worker_thread+0x1e8/0x3d0
<4> [99.591740] ? __pfx_worker_thread+0x10/0x10
<4> [99.591742] kthread+0x11f/0x250
<4> [99.591746] ? __pfx_kthread+0x10/0x10
<4> [99.591749] ret_from_fork+0x344/0x3a0
<4> [99.591751] ? __pfx_kthread+0x10/0x10
<4> [99.591754] ret_from_fork_asm+0x1a/0x30
<4> [99.591760] </TASK>
<4> [99.591761] irq event stamp: 197691
<4> [99.591762] hardirqs last enabled at (197697): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.591764] hardirqs last disabled at (197702): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.591771] softirqs last enabled at (196288): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.591774] softirqs last disabled at (196281): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.591776] ---[ end trace 0000000000000000 ]---
<6> [99.591778] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.591847] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.592647] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.592748] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.592885] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.592985] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.593248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.593331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.593417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.593500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.593581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.593661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.593742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.593832] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.593927] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.594038] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.595064] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.605772] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.606029] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.606892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.606967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.607040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.607115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.607188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.607259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.607329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.607398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.607468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.607539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.607610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.607679] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.607748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.607830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.607901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.607970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.608039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.608108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.608177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.608255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.608334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.608408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.608487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.608563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.608636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.608713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.608795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.608871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.608945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.609021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.609095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.609174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.609253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.609335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.609414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.609490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.609565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.609637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.609709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.609787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.609862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.609935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.610008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.610083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.610156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.610228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.610300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.610373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.610445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.610517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.610593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.610665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.610739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.610819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.610894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.610970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.611044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.611120] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.611404] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.611408] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4217, lrc_seqno=4217, guc_id=0, flags=0x73 in no process [-1]
<7> [99.611410] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.611467] ------------[ cut here ]------------
<4> [99.611468] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.611469] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.611538] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.611595] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.611600] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.611602] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.611603] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.611604] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.611609] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.611676] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.611677] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.611680] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.611681] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.611682] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.611683] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.611684] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.611685] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.611687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.611688] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.611689] PKRU: 55555554
<4> [99.611690] Call Trace:
<4> [99.611691] <TASK>
<4> [99.611696] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.611700] ? lock_release+0xce/0x280
<4> [99.611706] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.611711] process_one_work+0x22e/0x6b0
<4> [99.611716] worker_thread+0x1e8/0x3d0
<4> [99.611719] ? __pfx_worker_thread+0x10/0x10
<4> [99.611721] kthread+0x11f/0x250
<4> [99.611724] ? __pfx_kthread+0x10/0x10
<4> [99.611728] ret_from_fork+0x344/0x3a0
<4> [99.611730] ? __pfx_kthread+0x10/0x10
<4> [99.611732] ret_from_fork_asm+0x1a/0x30
<4> [99.611739] </TASK>
<4> [99.611740] irq event stamp: 202185
<4> [99.611741] hardirqs last enabled at (202191): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.611743] hardirqs last disabled at (202196): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.611745] softirqs last enabled at (200880): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.611748] softirqs last disabled at (200869): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.611750] ---[ end trace 0000000000000000 ]---
<5> [99.612724] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4222, lrc_seqno=4222, guc_id=0, flags=0x73 in no process [-1]
<7> [99.612729] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.612749] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.612815] ------------[ cut here ]------------
<4> [99.612817] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.612819] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:11/211
<4> [99.612904] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched
<7> [99.612868] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.612915] drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass
<7> [99.612935] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.612938] snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.612970] CPU: 6 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.612973] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.612974] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.612975] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.612980] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.613052] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.613054] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.613056] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.613057] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.613058] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.613059] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.613061] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.613062] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.613063] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.613064] CR2: 0000797c955494f8 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.613066] PKRU: 55555554
<4> [99.613067] Call Trace:
<4> [99.613068] <TASK>
<4> [99.613072] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.613077] ? lock_release+0xce/0x280
<4> [99.613083] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.613088] process_one_work+0x22e/0x6b0
<4> [99.613094] worker_thread+0x1e8/0x3d0
<4> [99.613096] ? __pfx_worker_thread+0x10/0x10
<4> [99.613099] kthread+0x11f/0x250
<4> [99.613102] ? __pfx_kthread+0x10/0x10
<4> [99.613105] ret_from_fork+0x344/0x3a0
<4> [99.613107] ? __pfx_kthread+0x10/0x10
<4> [99.613110] ret_from_fork_asm+0x1a/0x30
<4> [99.613117] </TASK>
<4> [99.613118] irq event stamp: 127995
<4> [99.613119] hardirqs last enabled at (128001): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.613121] hardirqs last disabled at (128006): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.613123] softirqs last enabled at (127226): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.613126] softirqs last disabled at (127219): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.613128] ---[ end trace 0000000000000000 ]---
<6> [99.613130] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.613200] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.613207] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.613255] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.613520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.613605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.613694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.613785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.613868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.613948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.614030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.614107] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.614198] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.614311] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.615324] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.625771] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.626025] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.626892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.626970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.627041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.627116] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.627192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.627265] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.627336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.627406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.627476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.627547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.627616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.627686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.627756] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.627840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.627910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.627981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.628055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.628126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.628196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.628274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.628350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.628423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.628499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.628574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.628647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.628721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.628807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.628884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.628958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.629032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.629107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.629189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.629270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.629349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.629427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.629504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.629578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.629654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.629728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.629812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.629889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.629962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.630034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.630110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.630184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.630256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.630327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.630400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.630470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.630541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.630614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.630688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.630761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.630845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.630920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.630991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.631062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.631135] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.631428] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.631432] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4222, lrc_seqno=4222, guc_id=0, flags=0x73 in no process [-1]
<7> [99.631435] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.631491] ------------[ cut here ]------------
<4> [99.631492] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.631493] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.631561] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.631618] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.631623] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.631625] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.631626] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.631627] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.631632] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.631701] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.631702] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.631704] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.631706] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.631707] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.631708] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.631709] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.631710] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.631711] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.631713] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.631714] PKRU: 55555554
<4> [99.631715] Call Trace:
<4> [99.631716] <TASK>
<4> [99.631721] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.631726] ? lock_release+0xce/0x280
<4> [99.631731] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.631736] process_one_work+0x22e/0x6b0
<4> [99.631741] worker_thread+0x1e8/0x3d0
<4> [99.631744] ? __pfx_worker_thread+0x10/0x10
<4> [99.631746] kthread+0x11f/0x250
<4> [99.631749] ? __pfx_kthread+0x10/0x10
<4> [99.631753] ret_from_fork+0x344/0x3a0
<4> [99.631755] ? __pfx_kthread+0x10/0x10
<4> [99.631757] ret_from_fork_asm+0x1a/0x30
<4> [99.631767] </TASK>
<4> [99.631768] irq event stamp: 206713
<4> [99.631770] hardirqs last enabled at (206719): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.631772] hardirqs last disabled at (206724): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.631774] softirqs last enabled at (205406): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.631777] softirqs last disabled at (205401): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.631780] ---[ end trace 0000000000000000 ]---
<6> [99.631781] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.631849] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.632652] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.632753] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.632882] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.632993] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.633257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.633342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.633431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.633516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.633599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.633680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.633767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.633848] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.633938] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.634050] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.635062] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.645769] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.646025] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.646892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.646969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.647040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.647114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.647187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.647260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.647332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.647403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.647474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.647546] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.647616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.647687] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.647757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.647842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.647913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.647983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.648053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.648124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.648194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.648271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.648348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.648422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.648499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.648574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.648649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.648725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.648828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.648905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.648982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.649058] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.649135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.649215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.649295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.649374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.649451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.649529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.649604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.649676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.649749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.649837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.649909] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.649980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.650054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.650130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.650204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.650277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.650348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.650420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.650491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.650564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.650641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.650714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.650792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.650882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.650955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.651026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.651097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.651173] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.651477] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.651480] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4222, lrc_seqno=4222, guc_id=0, flags=0x73 in no process [-1]
<7> [99.651483] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.651540] ------------[ cut here ]------------
<4> [99.651541] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.651542] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.651610] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.651667] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.651671] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.651674] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.651675] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.651676] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.651681] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.651747] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.651749] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.651751] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.651752] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.651753] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.651754] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.651755] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.651756] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.651758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.651759] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.651760] PKRU: 55555554
<4> [99.651764] Call Trace:
<4> [99.651765] <TASK>
<4> [99.651770] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.651775] ? lock_release+0xce/0x280
<4> [99.651780] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.651786] process_one_work+0x22e/0x6b0
<4> [99.651791] worker_thread+0x1e8/0x3d0
<4> [99.651794] ? __pfx_worker_thread+0x10/0x10
<4> [99.651796] kthread+0x11f/0x250
<4> [99.651799] ? __pfx_kthread+0x10/0x10
<4> [99.651803] ret_from_fork+0x344/0x3a0
<4> [99.651805] ? __pfx_kthread+0x10/0x10
<4> [99.651808] ret_from_fork_asm+0x1a/0x30
<4> [99.651814] </TASK>
<4> [99.651815] irq event stamp: 211211
<4> [99.651816] hardirqs last enabled at (211217): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.651819] hardirqs last disabled at (211222): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.651821] softirqs last enabled at (209898): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.651823] softirqs last disabled at (209891): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.651826] ---[ end trace 0000000000000000 ]---
<7> [99.652230] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.652361] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.652364] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.652731] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4224, lrc_seqno=4224, guc_id=0, flags=0x73 in no process [-1]
<7> [99.652734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.652871] ------------[ cut here ]------------
<4> [99.652872] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.652873] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#10: kworker/u64:7/207
<4> [99.653019] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.653097] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.653103] CPU: 10 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.653105] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.653107] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.653108] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [99.653132] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.653114] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.653209] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.653211] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.653213] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.653214] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.653215] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.653216] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.653217] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.653218] FS: 0000000000000000(0000) GS:ffff8888db1da000(0000) knlGS:0000000000000000
<4> [99.653220] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.653221] CR2: 00007973fe770000 CR3: 000000011d8e7001 CR4: 0000000000f72ef0
<4> [99.653222] PKRU: 55555554
<4> [99.653223] Call Trace:
<4> [99.653224] <TASK>
<4> [99.653229] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.653234] ? lock_release+0xce/0x280
<4> [99.653239] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.653244] process_one_work+0x22e/0x6b0
<4> [99.653250] worker_thread+0x1e8/0x3d0
<4> [99.653252] ? __pfx_worker_thread+0x10/0x10
<4> [99.653254] kthread+0x11f/0x250
<4> [99.653258] ? __pfx_kthread+0x10/0x10
<4> [99.653261] ret_from_fork+0x344/0x3a0
<4> [99.653263] ? __pfx_kthread+0x10/0x10
<4> [99.653266] ret_from_fork_asm+0x1a/0x30
<4> [99.653272] </TASK>
<4> [99.653273] irq event stamp: 212077
<4> [99.653274] hardirqs last enabled at (212083): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.653276] hardirqs last disabled at (212088): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.653279] softirqs last enabled at (209898): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.653281] softirqs last disabled at (209891): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.653283] ---[ end trace 0000000000000000 ]---
<6> [99.653285] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.653355] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.653402] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.653377] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.653476] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.653586] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<6> [99.653922] xe 0000:03:00.0: [drm] exec queue reset detected
<7> [99.654086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.654221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.654355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.654484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.654611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.654738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.654874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.655001] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.655138] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.655297] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.656673] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.667457] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.667782] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.668771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.668891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.669006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.669123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.669239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.669355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.669470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.669586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.669701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.669824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.669940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.670056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.670172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.670287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.670403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.670518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.670634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.670749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.670869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.670993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.671114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.671234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.671355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.671474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.671592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.671714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.671844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.671967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.672088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.672209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.672331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.672458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.672584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.672710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.672838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.672962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.673082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.673203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.673322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.673445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.673565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.673686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.673812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.673936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.674056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.674176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.674295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.674416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.674534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.674652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.674778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.674898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.675018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.675137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.675258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.675376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.675495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.675617] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.675995] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.677130] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4224, lrc_seqno=4224, guc_id=0, flags=0x73 in no process [-1]
<7> [99.677135] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.677235] ------------[ cut here ]------------
<4> [99.677237] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.677239] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#12: kworker/u64:7/207
<4> [99.677347] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.677432] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.677439] CPU: 12 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.677443] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.677444] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.677446] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.677453] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.677561] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.677563] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.677566] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.677568] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.677569] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.677571] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.677573] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.677574] FS: 0000000000000000(0000) GS:ffff8888db2da000(0000) knlGS:0000000000000000
<4> [99.677576] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.677578] CR2: 0000797c955424f8 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.677580] PKRU: 55555554
<4> [99.677581] Call Trace:
<4> [99.677583] <TASK>
<4> [99.677592] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.677599] ? lock_release+0xce/0x280
<4> [99.677608] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.677616] process_one_work+0x22e/0x6b0
<4> [99.677626] worker_thread+0x1e8/0x3d0
<4> [99.677629] ? __pfx_worker_thread+0x10/0x10
<4> [99.677632] kthread+0x11f/0x250
<4> [99.677637] ? __pfx_kthread+0x10/0x10
<4> [99.677642] ret_from_fork+0x344/0x3a0
<4> [99.677644] ? __pfx_kthread+0x10/0x10
<4> [99.677648] ret_from_fork_asm+0x1a/0x30
<4> [99.677659] </TASK>
<4> [99.677660] irq event stamp: 216601
<4> [99.677662] hardirqs last enabled at (216607): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.677665] hardirqs last disabled at (216612): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.677668] softirqs last enabled at (215216): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.677671] softirqs last disabled at (215211): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.677674] ---[ end trace 0000000000000000 ]---
<6> [99.677676] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.677794] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<5> [99.677806] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967169, lrc_seqno=4294967169, guc_id=37, flags=0x0 in xe_exec_system_ [2823]
<7> [99.677811] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.677823] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<6> [99.677897] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.677971] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.678120] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.678387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.678474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.678565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.678649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.678731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.678849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.678933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.679011] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.679102] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.679216] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.680232] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.690774] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.691031] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.691888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.691963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.692034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.692105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.692175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.692248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.692320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.692390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.692460] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.692532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.692604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.692676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.692750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.692843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.692924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.692994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.693065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.693136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.693205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.693282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.693358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.693434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.693510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.693586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.693660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.693735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.693828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.693914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.693989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.694064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.694139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.694218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.694297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.694375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.694453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.694533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.694607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.694681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.694754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.694849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.694937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.695014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.695087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.695165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.695239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.695313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.695385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.695458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.695531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.695602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.695677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.695750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.695839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.695925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.696003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.696076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.696150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.696225] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.696530] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.696535] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4224, lrc_seqno=4224, guc_id=0, flags=0x73 in no process [-1]
<7> [99.696537] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.696595] ------------[ cut here ]------------
<4> [99.696596] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.696598] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:8/208
<4> [99.696666] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.696726] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.696731] CPU: 6 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.696734] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.696735] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.696736] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.696741] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.696825] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.696828] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.696832] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.696834] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.696836] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.696838] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.696840] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.696842] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [99.696845] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.696847] CR2: 0000797c955494f8 CR3: 0000000114804003 CR4: 0000000000f72ef0
<4> [99.696849] PKRU: 55555554
<4> [99.696851] Call Trace:
<4> [99.696853] <TASK>
<4> [99.696863] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.696871] ? lock_release+0xce/0x280
<4> [99.696882] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.696891] process_one_work+0x22e/0x6b0
<4> [99.696901] worker_thread+0x1e8/0x3d0
<4> [99.696906] ? __pfx_worker_thread+0x10/0x10
<4> [99.696910] kthread+0x11f/0x250
<4> [99.696915] ? __pfx_kthread+0x10/0x10
<4> [99.696921] ret_from_fork+0x344/0x3a0
<4> [99.696924] ? __pfx_kthread+0x10/0x10
<4> [99.696929] ret_from_fork_asm+0x1a/0x30
<4> [99.696942] </TASK>
<4> [99.696943] irq event stamp: 579685
<4> [99.696945] hardirqs last enabled at (579691): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.696950] hardirqs last disabled at (579696): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.696953] softirqs last enabled at (578280): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.696957] softirqs last disabled at (578273): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.696960] ---[ end trace 0000000000000000 ]---
<5> [99.698017] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4234, lrc_seqno=4234, guc_id=0, flags=0x73 in no process [-1]
<7> [99.698022] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.698086] ------------[ cut here ]------------
<4> [99.698087] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<7> [99.698056] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.698088] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:7/207
<4> [99.698162] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801
<7> [99.698158] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.698205] idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.698226] CPU: 0 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.698228] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.698229] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.698230] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.698235] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.698304] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.698306] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.698308] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.698309] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.698310] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.698312] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.698313] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.698314] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.698315] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.698316] CR2: 0000797c95542458 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.698318] PKRU: 55555554
<4> [99.698319] Call Trace:
<4> [99.698320] <TASK>
<4> [99.698325] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.698330] ? lock_release+0xce/0x280
<4> [99.698336] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.698340] process_one_work+0x22e/0x6b0
<4> [99.698346] worker_thread+0x1e8/0x3d0
<4> [99.698349] ? __pfx_worker_thread+0x10/0x10
<4> [99.698351] kthread+0x11f/0x250
<4> [99.698354] ? __pfx_kthread+0x10/0x10
<4> [99.698357] ret_from_fork+0x344/0x3a0
<4> [99.698360] ? __pfx_kthread+0x10/0x10
<4> [99.698362] ret_from_fork_asm+0x1a/0x30
<4> [99.698369] </TASK>
<4> [99.698370] irq event stamp: 218021
<4> [99.698371] hardirqs last enabled at (218027): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.698373] hardirqs last disabled at (218032): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.698376] softirqs last enabled at (216652): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.698378] softirqs last disabled at (216641): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.698380] ---[ end trace 0000000000000000 ]---
<6> [99.698382] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.698451] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.698457] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.698524] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.698802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.698889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.698978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.699061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.699142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.699221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.699302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.699382] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.699469] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.699579] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.700588] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.710767] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.711022] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.711891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.711967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.712039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.712112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.712184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.712255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.712326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.712396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.712468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.712540] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.712611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.712683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.712752] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.712848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.712919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.712989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.713059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.713127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.713197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.713274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.713354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.713431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.713509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.713583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.713656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.713730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.713819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.713894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.713968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.714043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.714116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.714194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.714272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.714351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.714427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.714501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.714573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.714649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.714720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.714807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.714883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.714959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.715033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.715111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.715184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.715257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.715329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.715403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.715475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.715546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.715620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.715695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.715772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.715846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.715920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.715992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.716064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.716141] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.716448] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.716452] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4234, lrc_seqno=4234, guc_id=0, flags=0x73 in no process [-1]
<7> [99.716454] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.716512] ------------[ cut here ]------------
<4> [99.716513] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.716514] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:8/208
<4> [99.716583] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.716641] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.716646] CPU: 7 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.716648] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.716650] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.716651] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.716655] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.716722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.716724] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.716726] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.716727] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.716728] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.716729] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.716730] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.716732] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.716733] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.716734] CR2: 00007973fe774000 CR3: 000000011d8e7005 CR4: 0000000000f72ef0
<4> [99.716735] PKRU: 55555554
<4> [99.716736] Call Trace:
<4> [99.716737] <TASK>
<4> [99.716743] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.716748] ? lock_release+0xce/0x280
<4> [99.716753] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.716762] process_one_work+0x22e/0x6b0
<4> [99.716768] worker_thread+0x1e8/0x3d0
<4> [99.716771] ? __pfx_worker_thread+0x10/0x10
<4> [99.716773] kthread+0x11f/0x250
<4> [99.716777] ? __pfx_kthread+0x10/0x10
<4> [99.716780] ret_from_fork+0x344/0x3a0
<4> [99.716782] ? __pfx_kthread+0x10/0x10
<4> [99.716785] ret_from_fork_asm+0x1a/0x30
<4> [99.716792] </TASK>
<4> [99.716793] irq event stamp: 584307
<4> [99.716794] hardirqs last enabled at (584313): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.716797] hardirqs last disabled at (584318): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.716799] softirqs last enabled at (582794): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.716801] softirqs last disabled at (582787): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.716803] ---[ end trace 0000000000000000 ]---
<6> [99.716805] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.716875] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.717679] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.717799] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.717909] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.718013] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.718277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.718362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.718449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.718532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.718615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.718697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.718785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.718865] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.718958] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.719070] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.720074] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.730764] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.731018] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.732034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.732112] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.732183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.732256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.732329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.732401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.732470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.732539] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.732609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.732678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.732748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.732833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.732903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.732973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.733043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.733113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.733183] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.733253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.733326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.733406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.733485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.733559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.733635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.733708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.733786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.733860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.733939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.734016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.734092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.734167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.734244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.734324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.734402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.734481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.734558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.734634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.734710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.734789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.734863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.734941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.735017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.735089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.735161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.735237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.735311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.735383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.735454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.735530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.735605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.735676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.735751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.735838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.735912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.735982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.736054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.736126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.736199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.736274] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.736570] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.736574] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4234, lrc_seqno=4234, guc_id=0, flags=0x73 in no process [-1]
<7> [99.736577] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.736634] ------------[ cut here ]------------
<4> [99.736635] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.736636] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:8/208
<4> [99.736706] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.736768] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.736772] CPU: 7 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.736775] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.736776] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.736777] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.736782] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.736849] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.736850] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.736852] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.736853] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.736855] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.736856] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.736857] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.736858] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.736859] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.736860] CR2: 00007973fe774000 CR3: 000000011d8e7005 CR4: 0000000000f72ef0
<4> [99.736862] PKRU: 55555554
<4> [99.736863] Call Trace:
<4> [99.736864] <TASK>
<4> [99.736869] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.736873] ? lock_release+0xce/0x280
<4> [99.736879] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.736884] process_one_work+0x22e/0x6b0
<4> [99.736890] worker_thread+0x1e8/0x3d0
<4> [99.736892] ? __pfx_worker_thread+0x10/0x10
<4> [99.736895] kthread+0x11f/0x250
<4> [99.736898] ? __pfx_kthread+0x10/0x10
<4> [99.736901] ret_from_fork+0x344/0x3a0
<4> [99.736903] ? __pfx_kthread+0x10/0x10
<4> [99.736906] ret_from_fork_asm+0x1a/0x30
<4> [99.736912] </TASK>
<4> [99.736913] irq event stamp: 588803
<4> [99.736914] hardirqs last enabled at (588809): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.736917] hardirqs last disabled at (588814): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.736919] softirqs last enabled at (587482): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.736921] softirqs last disabled at (587469): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.736923] ---[ end trace 0000000000000000 ]---
<7> [99.737855] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<5> [99.737986] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4237, lrc_seqno=4237, guc_id=0, flags=0x73 in no process [-1]
<7> [99.737990] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<7> [99.737978] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.738057] ------------[ cut here ]------------
<4> [99.738058] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.738060] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.738134] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal
<7> [99.738151] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.738155] intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.738202] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.738205] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.738206] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.738207] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.738211] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.738281] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.738283] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.738285] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.738286] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.738287] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.738288] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.738289] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.738291] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.738292] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.738293] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.738295] PKRU: 55555554
<4> [99.738296] Call Trace:
<4> [99.738297] <TASK>
<4> [99.738302] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.738306] ? lock_release+0xce/0x280
<4> [99.738312] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.738317] process_one_work+0x22e/0x6b0
<4> [99.738323] worker_thread+0x1e8/0x3d0
<4> [99.738325] ? __pfx_worker_thread+0x10/0x10
<4> [99.738327] kthread+0x11f/0x250
<4> [99.738330] ? __pfx_kthread+0x10/0x10
<4> [99.738334] ret_from_fork+0x344/0x3a0
<4> [99.738336] ? __pfx_kthread+0x10/0x10
<4> [99.738338] ret_from_fork_asm+0x1a/0x30
<4> [99.738345] </TASK>
<4> [99.738346] irq event stamp: 1597821
<4> [99.738347] hardirqs last enabled at (1597827): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.738349] hardirqs last disabled at (1597832): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.738351] softirqs last enabled at (1596976): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.738354] softirqs last disabled at (1596971): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.738356] ---[ end trace 0000000000000000 ]---
<6> [99.738358] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.738427] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.738434] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.738485] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.738752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.738855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.738944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.739029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.739111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.739192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.739274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.739355] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.739450] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.739563] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.740568] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.750769] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.751024] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.751895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.751973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.752047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.752118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.752189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.752259] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.752331] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.752402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.752472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.752541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.752611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.752680] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.752749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.752836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.752913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.753008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.753090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.753168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.753237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.753316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.753393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.753468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.753547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.753621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.753693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.753772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.753851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.753926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.753998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.754072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.754145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.754225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.754302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.754380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.754456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.754529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.754601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.754674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.754745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.754833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.754906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.754979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.755052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.755129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.755204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.755277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.755364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.755438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.755510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.755581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.755656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.755728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.755814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.755887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.755961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.756032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.756103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.756178] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.756480] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.756484] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4237, lrc_seqno=4237, guc_id=0, flags=0x73 in no process [-1]
<7> [99.756486] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.756544] ------------[ cut here ]------------
<4> [99.756545] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.756546] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.756615] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.756672] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.756677] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.756680] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.756681] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.756682] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.756687] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.756759] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.756761] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.756763] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.756765] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.756766] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.756767] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.756769] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.756770] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.756771] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.756773] CR2: 00007973fe772000 CR3: 000000011d8e7006 CR4: 0000000000f72ef0
<4> [99.756774] PKRU: 55555554
<4> [99.756775] Call Trace:
<4> [99.756776] <TASK>
<4> [99.756781] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.756786] ? lock_release+0xce/0x280
<4> [99.756791] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.756796] process_one_work+0x22e/0x6b0
<4> [99.756802] worker_thread+0x1e8/0x3d0
<4> [99.756805] ? __pfx_worker_thread+0x10/0x10
<4> [99.756807] kthread+0x11f/0x250
<4> [99.756810] ? __pfx_kthread+0x10/0x10
<4> [99.756814] ret_from_fork+0x344/0x3a0
<4> [99.756816] ? __pfx_kthread+0x10/0x10
<4> [99.756819] ret_from_fork_asm+0x1a/0x30
<4> [99.756825] </TASK>
<4> [99.756826] irq event stamp: 593309
<4> [99.756827] hardirqs last enabled at (593315): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.756830] hardirqs last disabled at (593320): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.756832] softirqs last enabled at (591996): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.756835] softirqs last disabled at (591985): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.756837] ---[ end trace 0000000000000000 ]---
<6> [99.756839] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.756907] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.757709] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.757809] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.757910] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.758011] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.758274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.758359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.758446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.758526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.758607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.758687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.758773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.758850] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.758940] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.759051] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.760058] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.770764] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.771020] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.771885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.771961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.772032] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.772104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.772176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.772247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.772317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.772387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.772456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.772526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.772594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.772664] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.772734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.772814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.772885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.772960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.773031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.773101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.773170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.773247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.773324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.773400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.773477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.773550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.773624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.773699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.773784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.773862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.773937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.774012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.774088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.774168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.774249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.774331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.774408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.774484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.774556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.774630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.774703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.774783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.774857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.774930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.775002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.775078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.775150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.775222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.775293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.775367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.775440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.775514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.775590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.775662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.775733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.775825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.775900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.775989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.776060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.776134] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.776436] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.776439] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4237, lrc_seqno=4237, guc_id=0, flags=0x73 in no process [-1]
<7> [99.776442] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.776499] ------------[ cut here ]------------
<4> [99.776500] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.776501] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.776570] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.776626] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.776631] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.776633] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.776634] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.776635] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.776640] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.776708] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.776709] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.776711] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.776712] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.776713] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.776714] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.776715] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.776717] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.776718] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.776719] CR2: 00007973fe772000 CR3: 000000011d8e7006 CR4: 0000000000f72ef0
<4> [99.776721] PKRU: 55555554
<4> [99.776722] Call Trace:
<4> [99.776723] <TASK>
<4> [99.776727] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.776732] ? lock_release+0xce/0x280
<4> [99.776738] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.776742] process_one_work+0x22e/0x6b0
<4> [99.776748] worker_thread+0x1e8/0x3d0
<4> [99.776751] ? __pfx_worker_thread+0x10/0x10
<4> [99.776761] kthread+0x11f/0x250
<4> [99.776765] ? __pfx_kthread+0x10/0x10
<4> [99.776768] ret_from_fork+0x344/0x3a0
<4> [99.776770] ? __pfx_kthread+0x10/0x10
<4> [99.776773] ret_from_fork_asm+0x1a/0x30
<4> [99.776780] </TASK>
<4> [99.776781] irq event stamp: 597785
<4> [99.776782] hardirqs last enabled at (597791): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.776784] hardirqs last disabled at (597796): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.776786] softirqs last enabled at (597748): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.776789] softirqs last disabled at (597737): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.776791] ---[ end trace 0000000000000000 ]---
<7> [99.777354] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.777643] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4241, lrc_seqno=4241, guc_id=0, flags=0x73 in no process [-1]
<7> [99.777646] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.777708] ------------[ cut here ]------------
<4> [99.777709] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.777710] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:8/208
<4> [99.777789] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.777847] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.777852] CPU: 8 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.777854] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.777855] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.777856] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.777860] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.777927] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.777928] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.777930] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.777931] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.777932] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.777933] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.777934] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.777935] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.777936] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.777938] CR2: 00007973fe772000 CR3: 000000011d8e7006 CR4: 0000000000f72ef0
<4> [99.777939] PKRU: 55555554
<4> [99.777940] Call Trace:
<4> [99.777941] <TASK>
<4> [99.777945] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.777950] ? lock_release+0xce/0x280
<4> [99.777955] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.777960] process_one_work+0x22e/0x6b0
<4> [99.777966] worker_thread+0x1e8/0x3d0
<4> [99.777968] ? __pfx_worker_thread+0x10/0x10
<4> [99.777970] kthread+0x11f/0x250
<4> [99.777974] ? __pfx_kthread+0x10/0x10
<4> [99.777977] ret_from_fork+0x344/0x3a0
<4> [99.777979] ? __pfx_kthread+0x10/0x10
<4> [99.777982] ret_from_fork_asm+0x1a/0x30
<4> [99.777988] </TASK>
<4> [99.777989] irq event stamp: 598665
<4> [99.777990] hardirqs last enabled at (598671): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.777992] hardirqs last disabled at (598676): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.777994] softirqs last enabled at (597916): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.777996] softirqs last disabled at (597909): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.777998] ---[ end trace 0000000000000000 ]---
<6> [99.778000] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.778067] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.778073] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.778089] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.778094] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.778211] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.778331] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.778601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.778689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.778790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.778876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.778959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.779041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.779125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.779206] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.779300] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.779415] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.780430] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.790761] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.791039] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.791925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.792003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.792083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.792188] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.792262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.792336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.792408] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.792479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.792550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.792620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.792690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.792777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.792848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.792918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.792987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.793056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.793125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.793194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.793263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.793340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.793419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.793496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.793572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.793646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.793719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.793809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.793898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.793976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.794052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.794127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.794202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.794281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.794358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.794437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.794514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.794591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.794663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.794734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.794820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.794894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.794968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.795040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.795110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.795187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.795259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.795332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.795405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.795480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.795552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.795623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.795696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.795772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.795846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.795918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.795991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.796063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.796137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.796213] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.796539] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.796543] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4241, lrc_seqno=4241, guc_id=0, flags=0x73 in no process [-1]
<7> [99.796546] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.796604] ------------[ cut here ]------------
<4> [99.796605] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.796606] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [99.796677] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.796734] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.796739] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.796742] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.796743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.796744] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.796749] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.796837] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.796839] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.796841] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.796843] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.796844] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.796845] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.796846] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.796848] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.796850] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.796851] CR2: 00007973fe775000 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.796853] PKRU: 55555554
<4> [99.796854] Call Trace:
<4> [99.796855] <TASK>
<4> [99.796860] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.796865] ? lock_release+0xce/0x280
<4> [99.796870] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.796875] process_one_work+0x22e/0x6b0
<4> [99.796881] worker_thread+0x1e8/0x3d0
<4> [99.796884] ? __pfx_worker_thread+0x10/0x10
<4> [99.796886] kthread+0x11f/0x250
<4> [99.796889] ? __pfx_kthread+0x10/0x10
<4> [99.796893] ret_from_fork+0x344/0x3a0
<4> [99.796895] ? __pfx_kthread+0x10/0x10
<4> [99.796897] ret_from_fork_asm+0x1a/0x30
<4> [99.796904] </TASK>
<4> [99.796907] irq event stamp: 135847
<4> [99.796909] hardirqs last enabled at (135853): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.796911] hardirqs last disabled at (135858): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.796914] softirqs last enabled at (134328): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.796916] softirqs last disabled at (134321): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.796919] ---[ end trace 0000000000000000 ]---
<6> [99.796920] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.796989] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.797797] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.797901] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.798016] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.798143] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.798414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.798500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.798590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.798674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.798800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.798888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.798970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.799048] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.799139] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.799251] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.800262] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.810761] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.811016] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.811892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.811968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.812041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.812114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.812186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.812256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.812325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.812394] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.812464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.812534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.812607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.812676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.812746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.812838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.812908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.812978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.813047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.813117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.813185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.813262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.813338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.813413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.813490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.813563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.813636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.813711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.813797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.813872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.813946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.814021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.814095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.814175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.814262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.814343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.814420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.814499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.814575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.814649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.814749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.814878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.814967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.815040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.815115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.815191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.815263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.815336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.815408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.815480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.815552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.815623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.815699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.815780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.815853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.815923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.815997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.816069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.816141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.816218] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.816510] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.816514] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4241, lrc_seqno=4241, guc_id=0, flags=0x73 in no process [-1]
<7> [99.816516] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.816574] ------------[ cut here ]------------
<4> [99.816575] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.816576] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [99.816644] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.816702] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.816706] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.816709] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.816710] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.816712] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.816716] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.816807] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.816809] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.816811] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.816812] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.816814] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.816815] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.816816] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.816817] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.816818] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.816820] CR2: 00007973fe775000 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.816821] PKRU: 55555554
<4> [99.816822] Call Trace:
<4> [99.816824] <TASK>
<4> [99.816829] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.816834] ? lock_release+0xce/0x280
<4> [99.816840] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.816845] process_one_work+0x22e/0x6b0
<4> [99.816851] worker_thread+0x1e8/0x3d0
<4> [99.816854] ? __pfx_worker_thread+0x10/0x10
<4> [99.816856] kthread+0x11f/0x250
<4> [99.816859] ? __pfx_kthread+0x10/0x10
<4> [99.816863] ret_from_fork+0x344/0x3a0
<4> [99.816865] ? __pfx_kthread+0x10/0x10
<4> [99.816868] ret_from_fork_asm+0x1a/0x30
<4> [99.816874] </TASK>
<4> [99.816875] irq event stamp: 140323
<4> [99.816876] hardirqs last enabled at (140329): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.816879] hardirqs last disabled at (140334): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.816881] softirqs last enabled at (138716): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.816884] softirqs last disabled at (138711): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.816886] ---[ end trace 0000000000000000 ]---
<7> [99.817305] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.817398] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.817785] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4245, lrc_seqno=4245, guc_id=0, flags=0x73 in no process [-1]
<7> [99.817788] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.817851] ------------[ cut here ]------------
<4> [99.817852] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.817853] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [99.817923] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.817980] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.817985] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.817987] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.817988] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.817989] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.817994] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.818062] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.818063] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.818065] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.818066] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.818068] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.818069] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.818070] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.818071] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.818072] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.818073] CR2: 00007973fe775000 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [99.818074] PKRU: 55555554
<4> [99.818075] Call Trace:
<4> [99.818076] <TASK>
<4> [99.818081] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.818085] ? lock_release+0xce/0x280
<4> [99.818091] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.818096] process_one_work+0x22e/0x6b0
<4> [99.818101] worker_thread+0x1e8/0x3d0
<4> [99.818104] ? __pfx_worker_thread+0x10/0x10
<4> [99.818106] kthread+0x11f/0x250
<4> [99.818109] ? __pfx_kthread+0x10/0x10
<4> [99.818112] ret_from_fork+0x344/0x3a0
<4> [99.818114] ? __pfx_kthread+0x10/0x10
<4> [99.818117] ret_from_fork_asm+0x1a/0x30
<4> [99.818123] </TASK>
<4> [99.818124] irq event stamp: 141193
<4> [99.818125] hardirqs last enabled at (141199): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.818127] hardirqs last disabled at (141204): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.818129] softirqs last enabled at (138716): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.818132] softirqs last disabled at (138711): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.818134] ---[ end trace 0000000000000000 ]---
<6> [99.818135] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.818202] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.818208] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.818221] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.818337] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.818452] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.818724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.818825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.818917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.819001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.819085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.819165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.819246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.819327] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.819417] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.819527] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.820539] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.830760] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.831046] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.831927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.832003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.832075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.832147] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.832220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.832292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.832362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.832434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.832504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.832573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.832644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.832714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.832803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.832873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.832942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.833012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.833081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.833149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.833219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.833297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.833375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.833453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.833529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.833603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.833674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.833747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.833840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.833917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.833991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.834064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.834138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.834220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.834300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.834380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.834457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.834535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.834607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.834679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.834754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.834827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.834901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.834973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.835044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.835120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.835194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.835266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.835337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.835411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.835482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.835553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.835629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.835702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.835804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.835877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.835949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.836022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.836092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.836166] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.836457] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.837262] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4245, lrc_seqno=4245, guc_id=0, flags=0x73 in no process [-1]
<7> [99.837265] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.837323] ------------[ cut here ]------------
<4> [99.837325] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.837326] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:7/207
<4> [99.837402] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.837460] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.837465] CPU: 4 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.837468] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.837469] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.837470] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.837475] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.837543] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.837545] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.837547] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.837548] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.837549] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.837550] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.837551] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.837553] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [99.837554] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.837555] CR2: 0000797c9505f1af CR3: 000000015481e006 CR4: 0000000000f72ef0
<4> [99.837557] PKRU: 55555554
<4> [99.837558] Call Trace:
<4> [99.837559] <TASK>
<4> [99.837564] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.837569] ? lock_release+0xce/0x280
<4> [99.837574] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.837579] process_one_work+0x22e/0x6b0
<4> [99.837585] worker_thread+0x1e8/0x3d0
<4> [99.837587] ? __pfx_worker_thread+0x10/0x10
<4> [99.837589] kthread+0x11f/0x250
<4> [99.837592] ? __pfx_kthread+0x10/0x10
<4> [99.837596] ret_from_fork+0x344/0x3a0
<4> [99.837598] ? __pfx_kthread+0x10/0x10
<4> [99.837601] ret_from_fork_asm+0x1a/0x30
<4> [99.837607] </TASK>
<4> [99.837608] irq event stamp: 224559
<4> [99.837609] hardirqs last enabled at (224565): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.837611] hardirqs last disabled at (224570): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.837613] softirqs last enabled at (223044): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.837616] softirqs last disabled at (223035): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.837618] ---[ end trace 0000000000000000 ]---
<6> [99.837620] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.837688] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<7> [99.837703] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.837809] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [99.837904] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.837951] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.838215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.838301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.838388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.838472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.838554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.838635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.838718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.838806] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.838901] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.839012] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.840022] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.849756] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.850012] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.850873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.850948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.851021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.851095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.851166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.851237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.851307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.851377] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.851447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.851517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.851587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.851661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.851733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.851814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.851885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.851955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.852025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.852094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.852166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.852245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.852323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.852398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.852473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.852547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.852621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.852696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.852776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.852851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.852926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.853001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.853075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.853155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.853234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.853312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.853389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.853463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.853537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.853610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.853686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.853769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.853843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.853917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.853989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.854065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.854138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.854212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.854284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.854356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.854427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.854498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.854573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.854646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.854719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.854798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.854872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.854943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.855016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.855091] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.855378] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.855813] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4245, lrc_seqno=4245, guc_id=0, flags=0x73 in no process [-1]
<7> [99.855822] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.856189] ------------[ cut here ]------------
<4> [99.856191] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.856193] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#5: kworker/u64:7/207
<4> [99.856270] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.856349] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.856356] CPU: 5 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.856360] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.856362] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.856364] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<7> [99.856318] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<4> [99.856372] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.856484] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.856486] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.856489] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.856491] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.856493] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.856494] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.856495] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.856497] FS: 0000000000000000(0000) GS:ffff8888daf5a000(0000) knlGS:0000000000000000
<4> [99.856499] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.856501] CR2: 00007973fe76a000 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.856504] PKRU: 55555554
<4> [99.856506] Call Trace:
<4> [99.856507] <TASK>
<4> [99.856517] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.856526] ? lock_release+0xce/0x280
<7> [99.856465] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<4> [99.856535] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.856543] process_one_work+0x22e/0x6b0
<4> [99.856552] worker_thread+0x1e8/0x3d0
<4> [99.856556] ? __pfx_worker_thread+0x10/0x10
<4> [99.856559] kthread+0x11f/0x250
<4> [99.856563] ? __pfx_kthread+0x10/0x10
<4> [99.856567] ret_from_fork+0x344/0x3a0
<4> [99.856570] ? __pfx_kthread+0x10/0x10
<4> [99.856574] ret_from_fork_asm+0x1a/0x30
<4> [99.856585] </TASK>
<4> [99.856587] irq event stamp: 229029
<4> [99.856588] hardirqs last enabled at (229035): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.856592] hardirqs last disabled at (229040): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.856595] softirqs last enabled at (227624): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.856598] softirqs last disabled at (227619): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.856601] ---[ end trace 0000000000000000 ]---
<5> [99.857031] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4250, lrc_seqno=4250, guc_id=0, flags=0x73 in no process [-1]
<7> [99.857034] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.857093] ------------[ cut here ]------------
<4> [99.857094] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.857095] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#5: kworker/u64:7/207
<4> [99.857163] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.857225] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.857230] CPU: 5 UID: 0 PID: 207 Comm: kworker/u64:7 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.857233] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.857234] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.857235] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.857239] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.857308] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.857309] RSP: 0018:ffffc90001573c98 EFLAGS: 00010246
<4> [99.857311] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.857312] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.857314] RBP: ffffc90001573da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.857315] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.857316] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.857317] FS: 0000000000000000(0000) GS:ffff8888daf5a000(0000) knlGS:0000000000000000
<4> [99.857318] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.857319] CR2: 00007973fe76a000 CR3: 0000000003448003 CR4: 0000000000f72ef0
<4> [99.857321] PKRU: 55555554
<4> [99.857322] Call Trace:
<4> [99.857323] <TASK>
<4> [99.857327] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.857332] ? lock_release+0xce/0x280
<4> [99.857337] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.857342] process_one_work+0x22e/0x6b0
<4> [99.857347] worker_thread+0x1e8/0x3d0
<4> [99.857350] ? __pfx_worker_thread+0x10/0x10
<4> [99.857352] kthread+0x11f/0x250
<4> [99.857355] ? __pfx_kthread+0x10/0x10
<4> [99.857358] ret_from_fork+0x344/0x3a0
<4> [99.857360] ? __pfx_kthread+0x10/0x10
<4> [99.857363] ret_from_fork_asm+0x1a/0x30
<4> [99.857369] </TASK>
<4> [99.857370] irq event stamp: 230155
<4> [99.857371] hardirqs last enabled at (230161): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.857374] hardirqs last disabled at (230166): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.857376] softirqs last enabled at (227624): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.857378] softirqs last disabled at (227619): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.857380] ---[ end trace 0000000000000000 ]---
<6> [99.857382] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.857451] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.857457] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.857506] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.857776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.857865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.857953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.858035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.858119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.858200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.858282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.858360] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.858452] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.858564] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.859574] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.869755] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.870036] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.870914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.870989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.871060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.871133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.871206] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.871279] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.871349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.871421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.871492] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.871563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.871636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.871710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.871806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.871876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.871947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.872017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.872087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.872157] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.872226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.872304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.872382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.872460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.872538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.872612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.872686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.872765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.872842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.872919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.872993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.873069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.873145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.873225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.873303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.873382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.873459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.873533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.873605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.873679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.873762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.873839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.873911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.873983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.874053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.874129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.874201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.874272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.874344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.874418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.874489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.874561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.874637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.874710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.874791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.874862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.874934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.875008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.875078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.875152] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.875438] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.875441] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4250, lrc_seqno=4250, guc_id=0, flags=0x73 in no process [-1]
<7> [99.875444] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.875501] ------------[ cut here ]------------
<4> [99.875502] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.875503] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:8/208
<4> [99.875573] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.875630] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.875635] CPU: 7 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.875638] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.875639] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.875640] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.875645] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.875713] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.875715] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.875716] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.875718] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.875719] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.875720] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.875721] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.875722] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.875724] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.875725] CR2: 00007973fe776000 CR3: 000000011d8e7005 CR4: 0000000000f72ef0
<4> [99.875726] PKRU: 55555554
<4> [99.875727] Call Trace:
<4> [99.875728] <TASK>
<4> [99.875733] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.875738] ? lock_release+0xce/0x280
<4> [99.875744] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.875752] process_one_work+0x22e/0x6b0
<4> [99.875758] worker_thread+0x1e8/0x3d0
<4> [99.875761] ? __pfx_worker_thread+0x10/0x10
<4> [99.875763] kthread+0x11f/0x250
<4> [99.875766] ? __pfx_kthread+0x10/0x10
<4> [99.875770] ret_from_fork+0x344/0x3a0
<4> [99.875772] ? __pfx_kthread+0x10/0x10
<4> [99.875775] ret_from_fork_asm+0x1a/0x30
<4> [99.875781] </TASK>
<4> [99.875782] irq event stamp: 604969
<4> [99.875783] hardirqs last enabled at (604975): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.875785] hardirqs last disabled at (604980): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.875788] softirqs last enabled at (603656): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.875790] softirqs last disabled at (603649): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.875792] ---[ end trace 0000000000000000 ]---
<6> [99.875794] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.875862] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.876663] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.876769] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.876890] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.876993] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.877257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.877341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.877429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.877514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.877596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.877677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.877760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.877838] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.877928] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.878038] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.879042] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.889755] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.890012] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.890871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.890948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.891021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.891093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.891164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.891235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.891307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.891378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.891448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.891520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.891589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.891660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.891732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.891815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.891887] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.891956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.892026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.892097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.892167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.892248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.892327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.892403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.892477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.892551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.892623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.892699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.892784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.892862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.892936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.893010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.893084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.893165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.893243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.893321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.893397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.893474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.893547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.893619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.893692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.893773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.893846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.893918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.893990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.894066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.894140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.894213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.894284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.894357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.894429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.894501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.894576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.894649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.894722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.894802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.894875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.894948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.895023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.895100] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.895380] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.895384] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4250, lrc_seqno=4250, guc_id=0, flags=0x73 in no process [-1]
<7> [99.895387] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.895444] ------------[ cut here ]------------
<4> [99.895445] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.895446] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:8/208
<4> [99.895514] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.895570] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.895575] CPU: 7 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.895577] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.895578] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.895579] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.895584] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.895651] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.895653] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.895655] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.895656] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.895657] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.895658] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.895659] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.895660] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.895662] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.895663] CR2: 00007973fe776000 CR3: 000000011d8e7005 CR4: 0000000000f72ef0
<4> [99.895665] PKRU: 55555554
<4> [99.895666] Call Trace:
<4> [99.895666] <TASK>
<4> [99.895671] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.895676] ? lock_release+0xce/0x280
<4> [99.895681] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.895686] process_one_work+0x22e/0x6b0
<4> [99.895692] worker_thread+0x1e8/0x3d0
<4> [99.895694] ? __pfx_worker_thread+0x10/0x10
<4> [99.895697] kthread+0x11f/0x250
<4> [99.895700] ? __pfx_kthread+0x10/0x10
<4> [99.895703] ret_from_fork+0x344/0x3a0
<4> [99.895705] ? __pfx_kthread+0x10/0x10
<4> [99.895708] ret_from_fork_asm+0x1a/0x30
<4> [99.895714] </TASK>
<4> [99.895715] irq event stamp: 609431
<4> [99.895716] hardirqs last enabled at (609437): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.895719] hardirqs last disabled at (609442): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.895721] softirqs last enabled at (607920): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.895723] softirqs last disabled at (607915): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.895725] ---[ end trace 0000000000000000 ]---
<5> [99.896586] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4252, lrc_seqno=4252, guc_id=0, flags=0x73 in no process [-1]
<7> [99.896588] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.896648] ------------[ cut here ]------------
<4> [99.896649] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.896650] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#7: kworker/u64:8/208
<4> [99.896718] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog
<7> [99.896716] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [99.896721] mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.896785] CPU: 7 UID: 0 PID: 208 Comm: kworker/u64:8 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.896787] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.896788] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.896789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.896794] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.896860] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.896862] RSP: 0018:ffffc900015bfc98 EFLAGS: 00010246
<4> [99.896864] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.896865] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.896866] RBP: ffffc900015bfda8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.896867] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.896868] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.896869] FS: 0000000000000000(0000) GS:ffff8888db05a000(0000) knlGS:0000000000000000
<4> [99.896870] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.896872] CR2: 00007973fe776000 CR3: 000000011d8e7005 CR4: 0000000000f72ef0
<4> [99.896873] PKRU: 55555554
<4> [99.896874] Call Trace:
<4> [99.896875] <TASK>
<4> [99.896879] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.896884] ? lock_release+0xce/0x280
<4> [99.896889] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.896894] process_one_work+0x22e/0x6b0
<4> [99.896900] worker_thread+0x1e8/0x3d0
<4> [99.896903] ? __pfx_worker_thread+0x10/0x10
<4> [99.896905] kthread+0x11f/0x250
<4> [99.896908] ? __pfx_kthread+0x10/0x10
<4> [99.896911] ret_from_fork+0x344/0x3a0
<4> [99.896913] ? __pfx_kthread+0x10/0x10
<4> [99.896916] ret_from_fork_asm+0x1a/0x30
<4> [99.896922] </TASK>
<4> [99.896923] irq event stamp: 610317
<4> [99.896924] hardirqs last enabled at (610323): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.896926] hardirqs last disabled at (610328): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.896928] softirqs last enabled at (607920): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.896930] softirqs last disabled at (607915): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.896933] ---[ end trace 0000000000000000 ]---
<6> [99.896934] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.897001] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.897007] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.897097] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.897035] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.897172] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.897292] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.897565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.897654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.897744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.897853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.897936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.898017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.898096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.898175] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.898265] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.898377] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.899390] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.909764] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.910045] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.910920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.910997] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.911071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.911144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.911215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.911285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.911355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.911425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.911495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.911567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.911638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.911709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.911808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.911877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.911946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.912034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.912143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.912252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.912365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.912481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.912564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.912641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.912717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.912803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.912878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.912955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.913033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.913108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.913183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.913259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.913335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.913415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.913493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.913571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.913648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.913723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.913805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.913879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.913950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.914024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.914098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.914169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.914240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.914314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.914385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.914457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.914529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.914602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.914674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.914749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.914823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.914894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.914965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.915035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.915108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.915178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.915248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.915323] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.915605] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.915610] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4252, lrc_seqno=4252, guc_id=0, flags=0x73 in no process [-1]
<7> [99.915612] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.915670] ------------[ cut here ]------------
<4> [99.915671] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.915672] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#9: kworker/u64:3/203
<4> [99.915742] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.915842] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.915847] CPU: 9 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.915850] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.915851] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.915852] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.915857] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.915925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.915926] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.915928] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.915929] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.915931] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.915932] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.915933] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.915934] FS: 0000000000000000(0000) GS:ffff8888db15a000(0000) knlGS:0000000000000000
<4> [99.915935] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.915936] CR2: 00007e610a5e61b4 CR3: 0000000003448006 CR4: 0000000000f72ef0
<4> [99.915938] PKRU: 55555554
<4> [99.915939] Call Trace:
<4> [99.915940] <TASK>
<4> [99.915945] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.915950] ? lock_release+0xce/0x280
<4> [99.915955] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.915960] process_one_work+0x22e/0x6b0
<4> [99.915966] worker_thread+0x1e8/0x3d0
<4> [99.915968] ? __pfx_worker_thread+0x10/0x10
<4> [99.915971] kthread+0x11f/0x250
<4> [99.915974] ? __pfx_kthread+0x10/0x10
<4> [99.915977] ret_from_fork+0x344/0x3a0
<4> [99.915979] ? __pfx_kthread+0x10/0x10
<4> [99.915982] ret_from_fork_asm+0x1a/0x30
<4> [99.915988] </TASK>
<4> [99.915989] irq event stamp: 1604747
<4> [99.915990] hardirqs last enabled at (1604753): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.915993] hardirqs last disabled at (1604758): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.915995] softirqs last enabled at (1603132): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.915998] softirqs last disabled at (1603127): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.916000] ---[ end trace 0000000000000000 ]---
<6> [99.916002] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.916067] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.916816] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.916988] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.917088] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.917191] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.917457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.917542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.917631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.917714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.917808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.917890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.917969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.918046] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.918138] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.918249] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.919258] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.929753] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.930011] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.930878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.930954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.931027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.931100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.931173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.931246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.931317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.931387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.931457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.931528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.931600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.931672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.931750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.931825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.931895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.931965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.932036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.932105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.932175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.932252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.932329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.932405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.932481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.932554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.932628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.932703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.932810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.932888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.932963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.933037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.933111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.933192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.933270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.933349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.933425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.933501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.933574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.933644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.933714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.933805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.933881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.933956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.934028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.934103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.934174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.934247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.934320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.934397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.934471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.934545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.934620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.934693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.934795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.934868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.934946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.935019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.935091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.935164] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.935447] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.935451] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4252, lrc_seqno=4252, guc_id=0, flags=0x73 in no process [-1]
<7> [99.935454] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.935513] ------------[ cut here ]------------
<4> [99.935514] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.935515] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:3/203
<4> [99.935584] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.935642] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.935647] CPU: 8 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.935650] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.935651] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.935652] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.935657] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.935723] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.935725] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.935727] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.935728] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.935729] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.935730] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.935731] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.935732] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.935734] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.935735] CR2: 00007973fe772000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.935736] PKRU: 55555554
<4> [99.935737] Call Trace:
<4> [99.935738] <TASK>
<4> [99.935750] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.935755] ? lock_release+0xce/0x280
<4> [99.935761] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.935766] process_one_work+0x22e/0x6b0
<4> [99.935773] worker_thread+0x1e8/0x3d0
<4> [99.935776] ? __pfx_worker_thread+0x10/0x10
<4> [99.935778] kthread+0x11f/0x250
<4> [99.935781] ? __pfx_kthread+0x10/0x10
<4> [99.935785] ret_from_fork+0x344/0x3a0
<4> [99.935792] ? __pfx_kthread+0x10/0x10
<4> [99.935795] ret_from_fork_asm+0x1a/0x30
<4> [99.935801] </TASK>
<4> [99.935803] irq event stamp: 1609283
<4> [99.935804] hardirqs last enabled at (1609289): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.935806] hardirqs last disabled at (1609294): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.935808] softirqs last enabled at (1607878): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.935810] softirqs last disabled at (1607871): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.935813] ---[ end trace 0000000000000000 ]---
<7> [99.936668] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [99.936673] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4258, lrc_seqno=4258, guc_id=0, flags=0x73 in no process [-1]
<7> [99.936676] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.936736] ------------[ cut here ]------------
<4> [99.936737] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.936738] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:3/203
<4> [99.936822] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.936879] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.936884] CPU: 8 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.936886] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.936887] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.936888] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.936892] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.936959] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.936961] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.936963] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.936964] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.936965] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.936966] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.936967] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.936968] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.936969] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.936971] CR2: 00007973fe772000 CR3: 000000015481e003 CR4: 0000000000f72ef0
<4> [99.936972] PKRU: 55555554
<4> [99.936973] Call Trace:
<4> [99.936974] <TASK>
<4> [99.936979] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.936983] ? lock_release+0xce/0x280
<4> [99.936988] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.936993] process_one_work+0x22e/0x6b0
<4> [99.936998] worker_thread+0x1e8/0x3d0
<4> [99.937001] ? __pfx_worker_thread+0x10/0x10
<4> [99.937003] kthread+0x11f/0x250
<4> [99.937006] ? __pfx_kthread+0x10/0x10
<4> [99.937010] ret_from_fork+0x344/0x3a0
<4> [99.937012] ? __pfx_kthread+0x10/0x10
<4> [99.937014] ret_from_fork_asm+0x1a/0x30
<4> [99.937021] </TASK>
<4> [99.937022] irq event stamp: 1610175
<4> [99.937023] hardirqs last enabled at (1610181): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.937025] hardirqs last disabled at (1610186): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.937027] softirqs last enabled at (1609426): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.937029] softirqs last disabled at (1609421): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.937031] ---[ end trace 0000000000000000 ]---
<6> [99.937033] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.937099] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.937115] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.937113] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [99.937169] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.937283] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.937402] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.937666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.937757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.937847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.937931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.938013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.938092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.938171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.938253] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.938346] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.938457] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.939466] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.950102] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.950360] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.951189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.951264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.951337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.951410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.951482] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.951553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.951624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.951693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.951772] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.951841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.951911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.951982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.952051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.952121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.952191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.952262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.952332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.952404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.952474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.952551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.952628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.952703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.952787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.952861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.952934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.953011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.953090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.953167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.953245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.953322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.953398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.953477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.953557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.953636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.953714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.953798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.953871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.953943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.954015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.954091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.954168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.954243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.954316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.954392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.954465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.954538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.954609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.954682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.954757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.954829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.954906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.954978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.955051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.955124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.955198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.955270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.955341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.955420] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.955713] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.955717] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4258, lrc_seqno=4258, guc_id=0, flags=0x73 in no process [-1]
<7> [99.955719] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.955792] ------------[ cut here ]------------
<4> [99.955793] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.955794] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:3/203
<4> [99.955865] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.955921] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.955926] CPU: 8 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.955928] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.955929] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.955930] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.955935] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.956002] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.956004] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.956005] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.956007] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.956008] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.956009] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.956010] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.956011] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.956012] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.956013] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.956015] PKRU: 55555554
<4> [99.956016] Call Trace:
<4> [99.956017] <TASK>
<4> [99.956022] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.956026] ? lock_release+0xce/0x280
<4> [99.956032] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.956037] process_one_work+0x22e/0x6b0
<4> [99.956042] worker_thread+0x1e8/0x3d0
<4> [99.956045] ? __pfx_worker_thread+0x10/0x10
<4> [99.956047] kthread+0x11f/0x250
<4> [99.956050] ? __pfx_kthread+0x10/0x10
<4> [99.956054] ret_from_fork+0x344/0x3a0
<4> [99.956056] ? __pfx_kthread+0x10/0x10
<4> [99.956058] ret_from_fork_asm+0x1a/0x30
<4> [99.956065] </TASK>
<4> [99.956066] irq event stamp: 1614693
<4> [99.956067] hardirqs last enabled at (1614699): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.956069] hardirqs last disabled at (1614704): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.956071] softirqs last enabled at (1613352): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.956074] softirqs last disabled at (1613345): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.956076] ---[ end trace 0000000000000000 ]---
<6> [99.956078] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.956145] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.956950] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.957050] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.957147] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.957249] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.957512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.957593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.957680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.957767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.957847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.957926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.958008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.958087] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.958178] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.958288] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.959293] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.969749] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.970004] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.970873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.970949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.971022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.971097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.971172] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.971245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.971316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.971387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.971457] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.971527] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.971597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.971668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.971739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.971825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.971896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.971965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.972035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.972104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.972174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.972253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.972329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.972405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.972482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.972558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.972632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.972709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.972797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.972873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.972946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.973022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.973097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.973176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.973254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.973331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.973407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.973483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.973555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.973627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.973698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.973782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.973856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.973928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.974003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.974080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.974156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.974228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.974299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.974373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.974449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.974523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.974600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.974675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.974753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.974826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.974902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.974976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.975050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.975127] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.975412] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.975415] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4258, lrc_seqno=4258, guc_id=0, flags=0x73 in no process [-1]
<7> [99.975417] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.975475] ------------[ cut here ]------------
<4> [99.975476] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.975477] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:3/203
<4> [99.975545] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.975602] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.975606] CPU: 8 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.975609] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.975610] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.975611] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.975615] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.975683] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.975685] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.975687] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.975688] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.975689] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.975690] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.975691] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.975692] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.975694] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.975695] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.975696] PKRU: 55555554
<4> [99.975697] Call Trace:
<4> [99.975698] <TASK>
<4> [99.975703] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.975707] ? lock_release+0xce/0x280
<4> [99.975713] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.975718] process_one_work+0x22e/0x6b0
<4> [99.975724] worker_thread+0x1e8/0x3d0
<4> [99.975726] ? __pfx_worker_thread+0x10/0x10
<4> [99.975728] kthread+0x11f/0x250
<4> [99.975732] ? __pfx_kthread+0x10/0x10
<4> [99.975735] ret_from_fork+0x344/0x3a0
<4> [99.975737] ? __pfx_kthread+0x10/0x10
<4> [99.975740] ret_from_fork_asm+0x1a/0x30
<4> [99.975749] </TASK>
<4> [99.975750] irq event stamp: 1619201
<4> [99.975751] hardirqs last enabled at (1619207): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.975754] hardirqs last disabled at (1619212): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.975756] softirqs last enabled at (1617888): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.975759] softirqs last disabled at (1617881): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.975761] ---[ end trace 0000000000000000 ]---
<5> [99.976588] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4262, lrc_seqno=4262, guc_id=0, flags=0x73 in no process [-1]
<7> [99.976591] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.976650] ------------[ cut here ]------------
<4> [99.976651] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.976652] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#8: kworker/u64:3/203
<4> [99.976719] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.976780] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.976784] CPU: 8 UID: 0 PID: 203 Comm: kworker/u64:3 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.976786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.976787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.976788] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.976793] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.976858] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.976859] RSP: 0018:ffffc900014c3c98 EFLAGS: 00010246
<4> [99.976861] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.976862] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.976863] RBP: ffffc900014c3da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.976864] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.976865] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.976867] FS: 0000000000000000(0000) GS:ffff8888db0da000(0000) knlGS:0000000000000000
<4> [99.976868] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.976869] CR2: 00007973fe772000 CR3: 0000000003448004 CR4: 0000000000f72ef0
<4> [99.976870] PKRU: 55555554
<4> [99.976871] Call Trace:
<4> [99.976872] <TASK>
<4> [99.976876] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.976881] ? lock_release+0xce/0x280
<4> [99.976886] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.976891] process_one_work+0x22e/0x6b0
<4> [99.976896] worker_thread+0x1e8/0x3d0
<4> [99.976899] ? __pfx_worker_thread+0x10/0x10
<4> [99.976901] kthread+0x11f/0x250
<4> [99.976904] ? __pfx_kthread+0x10/0x10
<4> [99.976907] ret_from_fork+0x344/0x3a0
<4> [99.976909] ? __pfx_kthread+0x10/0x10
<4> [99.976912] ret_from_fork_asm+0x1a/0x30
<4> [99.976919] </TASK>
<4> [99.976919] irq event stamp: 1620073
<4> [99.976920] hardirqs last enabled at (1620079): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.976923] hardirqs last disabled at (1620084): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.976925] softirqs last enabled at (1617888): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.976927] softirqs last disabled at (1617881): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.976929] ---[ end trace 0000000000000000 ]---
<6> [99.976931] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.976995] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.977001] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.977013] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.977128] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.977241] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.977512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.977602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.977692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.977788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.977870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.977950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.978032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.978112] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.978203] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.978315] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.979326] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [99.989750] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [99.990008] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [99.990871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [99.990948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [99.991019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [99.991091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [99.991160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [99.991229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [99.991300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [99.991370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [99.991438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [99.991507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [99.991577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [99.991648] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [99.991720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [99.991802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [99.991872] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [99.991942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [99.992011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [99.992082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [99.992151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [99.992230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [99.992306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [99.992381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [99.992458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [99.992530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [99.992604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [99.992680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [99.992767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [99.992843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [99.992918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [99.992993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [99.993069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [99.993149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [99.993232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [99.993313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [99.993391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [99.993467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [99.993539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [99.993611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [99.993683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [99.993769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [99.993842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [99.993914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [99.993986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [99.994062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [99.994135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [99.994206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [99.994277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [99.994350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [99.994420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [99.994491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [99.994564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [99.994636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [99.994708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [99.994804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [99.994877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [99.994947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [99.995018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [99.995093] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [99.995397] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [99.995401] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4262, lrc_seqno=4262, guc_id=0, flags=0x73 in no process [-1]
<7> [99.995404] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [99.995461] ------------[ cut here ]------------
<4> [99.995462] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [99.995463] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [99.995532] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [99.995591] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [99.995596] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [99.995599] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [99.995600] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [99.995601] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [99.995606] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [99.995672] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [99.995674] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [99.995676] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [99.995677] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [99.995679] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [99.995680] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [99.995681] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [99.995682] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [99.995683] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [99.995685] CR2: 0000797c95526778 CR3: 0000000194f43003 CR4: 0000000000f72ef0
<4> [99.995686] PKRU: 55555554
<4> [99.995687] Call Trace:
<4> [99.995688] <TASK>
<4> [99.995693] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [99.995698] ? lock_release+0xce/0x280
<4> [99.995703] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [99.995708] process_one_work+0x22e/0x6b0
<4> [99.995714] worker_thread+0x1e8/0x3d0
<4> [99.995717] ? __pfx_worker_thread+0x10/0x10
<4> [99.995719] kthread+0x11f/0x250
<4> [99.995722] ? __pfx_kthread+0x10/0x10
<4> [99.995725] ret_from_fork+0x344/0x3a0
<4> [99.995727] ? __pfx_kthread+0x10/0x10
<4> [99.995730] ret_from_fork_asm+0x1a/0x30
<4> [99.995736] </TASK>
<4> [99.995737] irq event stamp: 147257
<4> [99.995738] hardirqs last enabled at (147263): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [99.995746] hardirqs last disabled at (147270): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [99.995749] softirqs last enabled at (145952): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.995753] softirqs last disabled at (145897): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [99.995757] ---[ end trace 0000000000000000 ]---
<6> [99.995759] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [99.995837] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [99.996655] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [99.996761] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [99.996861] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [99.997176] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [99.997441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [99.997527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [99.997617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [99.997700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [99.997796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [99.997876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [99.997958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [99.998036] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [99.998127] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [99.998238] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [99.999247] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.009748] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.010002] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.010858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.010935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.011008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.011099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.011170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.011240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.011310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.011379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.011448] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.011517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.011587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.011659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.011729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.011818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.011888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.011957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.012027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.012097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.012166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.012245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.012321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.012396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.012471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.012543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.012616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.012690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.012779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.012853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.012927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.013000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.013075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.013155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.013233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.013311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.013388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.013465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.013539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.013613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.013685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.013768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.013844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.013915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.013987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.014063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.014134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.014206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.014277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.014349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.014421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.014493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.014568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.014641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.014713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.014800] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.014875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.014946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.015017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.015091] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.015391] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.016198] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4262, lrc_seqno=4262, guc_id=0, flags=0x73 in no process [-1]
<7> [100.016201] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.016260] ------------[ cut here ]------------
<4> [100.016261] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.016262] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [100.016330] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.016389] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.016394] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.016396] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.016398] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.016399] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.016404] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.016470] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.016471] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [100.016474] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.016475] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.016476] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.016477] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.016478] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.016479] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [100.016481] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.016482] CR2: 0000797c95526778 CR3: 0000000194f43003 CR4: 0000000000f72ef0
<4> [100.016483] PKRU: 55555554
<4> [100.016484] Call Trace:
<4> [100.016485] <TASK>
<4> [100.016490] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.016495] ? lock_release+0xce/0x280
<4> [100.016501] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.016506] process_one_work+0x22e/0x6b0
<4> [100.016512] worker_thread+0x1e8/0x3d0
<4> [100.016514] ? __pfx_worker_thread+0x10/0x10
<4> [100.016516] kthread+0x11f/0x250
<4> [100.016519] ? __pfx_kthread+0x10/0x10
<4> [100.016523] ret_from_fork+0x344/0x3a0
<4> [100.016525] ? __pfx_kthread+0x10/0x10
<4> [100.016527] ret_from_fork_asm+0x1a/0x30
<4> [100.016534] </TASK>
<4> [100.016535] irq event stamp: 151745
<4> [100.016536] hardirqs last enabled at (151751): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.016538] hardirqs last disabled at (151756): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.016540] softirqs last enabled at (150438): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.016543] softirqs last disabled at (150427): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.016545] ---[ end trace 0000000000000000 ]---
<7> [100.016590] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.016710] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [100.016951] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4264, lrc_seqno=4264, guc_id=0, flags=0x73 in no process [-1]
<7> [100.016957] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.017064] ------------[ cut here ]------------
<4> [100.017066] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.017067] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#13: kworker/u64:4/204
<4> [100.017183] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc
<7> [100.017188] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [100.017194] xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof
<7> [100.017224] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<4> [100.017228] kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.017278] CPU: 13 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.017282] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.017284] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.017286] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.017293] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.017403] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.017405] RSP: 0000:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.017408] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.017410] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.017412] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.017413] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.017415] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.017417] FS: 0000000000000000(0000) GS:ffff8888db35a000(0000) knlGS:0000000000000000
<4> [100.017419] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.017420] CR2: 0000797c955496d8 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.017422] PKRU: 55555554
<4> [100.017424] Call Trace:
<4> [100.017425] <TASK>
<4> [100.017434] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.017442] ? lock_release+0xce/0x280
<4> [100.017450] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.017459] process_one_work+0x22e/0x6b0
<4> [100.017468] worker_thread+0x1e8/0x3d0
<4> [100.017472] ? __pfx_worker_thread+0x10/0x10
<4> [100.017475] kthread+0x11f/0x250
<4> [100.017480] ? __pfx_kthread+0x10/0x10
<4> [100.017484] ret_from_fork+0x344/0x3a0
<4> [100.017487] ? __pfx_kthread+0x10/0x10
<4> [100.017491] ret_from_fork_asm+0x1a/0x30
<4> [100.017502] </TASK>
<4> [100.017503] irq event stamp: 21469
<4> [100.017505] hardirqs last enabled at (21475): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.017508] hardirqs last disabled at (21480): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.017511] softirqs last enabled at (20662): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.017514] softirqs last disabled at (20657): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.017517] ---[ end trace 0000000000000000 ]---
<6> [100.017519] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.017629] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.017638] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.017689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.018038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.018169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.018303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.018432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.018560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.018686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.018821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.018949] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.019085] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.019246] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.020619] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.031395] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.031710] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.032701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.032821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.032936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.033052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.033167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.033283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.033398] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.033513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.033629] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.033749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.033866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.033981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.034097] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.034213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.034328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.034444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.034559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.034674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.034796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.034917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.035038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.035157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.035278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.035396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.035515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.035636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.035768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.035892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.036013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.036136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.036258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.036385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.036511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.036637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.036768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.036892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.037013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.037133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.037252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.037376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.037497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.037617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.037740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.037865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.037985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.038106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.038225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.038345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.038464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.038583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.038705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.038830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.038951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.039070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.039191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.039309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.039428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.039550] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.039919] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.039925] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4264, lrc_seqno=4264, guc_id=0, flags=0x73 in no process [-1]
<7> [100.039927] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.039989] ------------[ cut here ]------------
<4> [100.039990] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.039991] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:4/204
<4> [100.040065] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.040122] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.040127] CPU: 4 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.040130] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.040131] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.040132] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.040137] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.040206] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.040208] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.040210] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.040212] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.040213] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.040214] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.040215] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.040216] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.040218] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.040219] CR2: 0000797c950d46be CR3: 000000015481e006 CR4: 0000000000f72ef0
<4> [100.040220] PKRU: 55555554
<4> [100.040221] Call Trace:
<4> [100.040222] <TASK>
<4> [100.040227] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.040232] ? lock_release+0xce/0x280
<4> [100.040238] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.040243] process_one_work+0x22e/0x6b0
<4> [100.040248] worker_thread+0x1e8/0x3d0
<4> [100.040251] ? __pfx_worker_thread+0x10/0x10
<4> [100.040253] kthread+0x11f/0x250
<4> [100.040257] ? __pfx_kthread+0x10/0x10
<4> [100.040260] ret_from_fork+0x344/0x3a0
<4> [100.040262] ? __pfx_kthread+0x10/0x10
<4> [100.040265] ret_from_fork_asm+0x1a/0x30
<4> [100.040271] </TASK>
<4> [100.040272] irq event stamp: 25983
<4> [100.040273] hardirqs last enabled at (25989): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.040275] hardirqs last disabled at (25994): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.040278] softirqs last enabled at (24864): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.040280] softirqs last disabled at (24855): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.040282] ---[ end trace 0000000000000000 ]---
<6> [100.040284] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.040352] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.040359] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.040407] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.040674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.040767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.040857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.040941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.041027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.041109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.041190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.041271] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.041367] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.041479] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.042489] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.053095] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.053344] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.054329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.054422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.054505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.054583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.054659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.054748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.054825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.054899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.054974] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.055050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.055125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.055200] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.055276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.055351] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.055424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.055499] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.055575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.055652] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.055728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.055827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.055912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.055992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.056074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.056153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.056232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.056314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.056398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.056482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.056564] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.056646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.056727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.056830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.056918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.057004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.057087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.057167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.057246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.057325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.057403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.057483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.057561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.057639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.057720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.057811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.057890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.057969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.058047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.058126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.058202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.058279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.058359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.058437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.058515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.058594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.058671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.058750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.058829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.058911] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.059248] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.060057] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4264, lrc_seqno=4264, guc_id=0, flags=0x73 in no process [-1]
<7> [100.060060] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.060123] ------------[ cut here ]------------
<4> [100.060124] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.060126] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:4/204
<4> [100.060207] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.060265] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.060269] CPU: 6 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.060272] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.060273] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.060274] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.060279] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.060353] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.060354] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.060356] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.060357] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.060359] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.060360] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.060361] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.060362] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [100.060363] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.060365] CR2: 0000797c950e1078 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [100.060366] PKRU: 55555554
<4> [100.060367] Call Trace:
<4> [100.060368] <TASK>
<4> [100.060373] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.060378] ? lock_release+0xce/0x280
<4> [100.060384] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.060389] process_one_work+0x22e/0x6b0
<4> [100.060394] worker_thread+0x1e8/0x3d0
<4> [100.060397] ? __pfx_worker_thread+0x10/0x10
<4> [100.060399] kthread+0x11f/0x250
<4> [100.060403] ? __pfx_kthread+0x10/0x10
<4> [100.060406] ret_from_fork+0x344/0x3a0
<4> [100.060408] ? __pfx_kthread+0x10/0x10
<4> [100.060411] ret_from_fork_asm+0x1a/0x30
<4> [100.060417] </TASK>
<4> [100.060418] irq event stamp: 30477
<4> [100.060419] hardirqs last enabled at (30483): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.060422] hardirqs last disabled at (30488): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.060424] softirqs last enabled at (29000): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.060426] softirqs last disabled at (28991): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.060429] ---[ end trace 0000000000000000 ]---
<7> [100.060473] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.060581] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [100.060692] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4269, lrc_seqno=4269, guc_id=0, flags=0x73 in no process [-1]
<7> [100.060694] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.060762] ------------[ cut here ]------------
<4> [100.060763] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.060764] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:4/204
<4> [100.060834] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.060890] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.060895] CPU: 6 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.060897] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.060898] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.060899] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.060904] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.060972] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.060974] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.060976] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.060977] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.060978] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.060979] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.060980] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.060981] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [100.060982] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.060984] CR2: 0000797c950e1078 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [100.060985] PKRU: 55555554
<4> [100.060986] Call Trace:
<4> [100.060987] <TASK>
<4> [100.060991] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.060996] ? lock_release+0xce/0x280
<4> [100.061001] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.061006] process_one_work+0x22e/0x6b0
<4> [100.061012] worker_thread+0x1e8/0x3d0
<4> [100.061014] ? __pfx_worker_thread+0x10/0x10
<4> [100.061016] kthread+0x11f/0x250
<4> [100.061019] ? __pfx_kthread+0x10/0x10
<4> [100.061023] ret_from_fork+0x344/0x3a0
<4> [100.061025] ? __pfx_kthread+0x10/0x10
<4> [100.061027] ret_from_fork_asm+0x1a/0x30
<4> [100.061034] </TASK>
<4> [100.061035] irq event stamp: 31365
<4> [100.061036] hardirqs last enabled at (31371): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.061038] hardirqs last disabled at (31376): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.061040] softirqs last enabled at (29000): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.061042] softirqs last disabled at (28991): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.061044] ---[ end trace 0000000000000000 ]---
<6> [100.061046] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<7> [100.061092] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<6> [100.061115] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.061125] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.061139] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.061402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.061487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.061573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.061656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.061741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.061820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.061900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.061979] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.062069] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.062180] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.063185] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.073797] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.074078] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.074912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.074996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.075075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.075153] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.075230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.075305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.075381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.075458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.075535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.075611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.075685] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.075770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.075845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.075919] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.075994] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.076069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.076144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.076219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.076293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.076378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.076462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.076545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.076626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.076705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.076790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.076871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.076955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.077036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.077115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.077196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.077277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.077361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.077444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.077528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.077609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.077690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.077775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.077853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.077931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.078012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.078092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.078171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.078251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.078335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.078412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.078490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.078566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.078646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.078724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.078811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.078892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.078970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.079050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.079127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.079205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.079283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.079360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.079441] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.079763] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.080565] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4269, lrc_seqno=4269, guc_id=0, flags=0x73 in no process [-1]
<7> [100.080568] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.080628] ------------[ cut here ]------------
<4> [100.080629] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.080630] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:4/204
<4> [100.080707] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.080770] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.080775] CPU: 6 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.080778] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.080779] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.080781] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.080785] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.080853] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.080854] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.080856] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.080858] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.080859] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.080860] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.080861] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.080862] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [100.080863] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.080865] CR2: 0000797c950e1078 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [100.080866] PKRU: 55555554
<4> [100.080867] Call Trace:
<4> [100.080868] <TASK>
<4> [100.080873] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.080878] ? lock_release+0xce/0x280
<4> [100.080884] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.080888] process_one_work+0x22e/0x6b0
<4> [100.080894] worker_thread+0x1e8/0x3d0
<4> [100.080897] ? __pfx_worker_thread+0x10/0x10
<4> [100.080899] kthread+0x11f/0x250
<4> [100.080902] ? __pfx_kthread+0x10/0x10
<4> [100.080905] ret_from_fork+0x344/0x3a0
<4> [100.080907] ? __pfx_kthread+0x10/0x10
<4> [100.080910] ret_from_fork_asm+0x1a/0x30
<4> [100.080916] </TASK>
<4> [100.080917] irq event stamp: 35881
<4> [100.080918] hardirqs last enabled at (35887): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.080921] hardirqs last disabled at (35892): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.080923] softirqs last enabled at (34350): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.080925] softirqs last disabled at (34343): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.080928] ---[ end trace 0000000000000000 ]---
<6> [100.080929] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.080997] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<7> [100.081013] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.081126] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<6> [100.081219] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.081267] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.081532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.081617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.081704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.081797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.081880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.081962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.082047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.082127] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.082219] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.082331] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.083335] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.093743] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.093998] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.094876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.094951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.095024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.095095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.095166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.095237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.095308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.095379] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.095449] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.095519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.095587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.095656] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.095726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.095812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.095883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.095952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.096022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.096090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.096158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.096235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.096310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.096384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.096460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.096533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.096606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.096682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.096768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.096846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.096925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.097004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.097081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.097162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.097242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.097321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.097397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.097473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.097546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.097620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.097693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.097777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.097853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.097928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.098001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.098077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.098149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.098222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.098292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.098365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.098435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.098505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.098578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.098649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.098722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.098806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.098878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.098950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.099021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.099095] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.099395] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.100197] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4269, lrc_seqno=4269, guc_id=0, flags=0x73 in no process [-1]
<7> [100.100200] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.100259] ------------[ cut here ]------------
<4> [100.100260] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.100261] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:4/204
<4> [100.100336] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.100394] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.100398] CPU: 6 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.100401] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.100402] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.100403] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.100408] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.100476] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.100478] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.100480] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.100481] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.100482] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.100483] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.100484] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.100485] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [100.100487] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.100488] CR2: 0000797c950e1078 CR3: 0000000114804003 CR4: 0000000000f72ef0
<4> [100.100489] PKRU: 55555554
<4> [100.100490] Call Trace:
<4> [100.100491] <TASK>
<4> [100.100496] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.100501] ? lock_release+0xce/0x280
<4> [100.100506] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.100511] process_one_work+0x22e/0x6b0
<4> [100.100517] worker_thread+0x1e8/0x3d0
<4> [100.100519] ? __pfx_worker_thread+0x10/0x10
<4> [100.100522] kthread+0x11f/0x250
<4> [100.100525] ? __pfx_kthread+0x10/0x10
<4> [100.100528] ret_from_fork+0x344/0x3a0
<4> [100.100530] ? __pfx_kthread+0x10/0x10
<4> [100.100533] ret_from_fork_asm+0x1a/0x30
<4> [100.100539] </TASK>
<4> [100.100540] irq event stamp: 40389
<4> [100.100541] hardirqs last enabled at (40395): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.100544] hardirqs last disabled at (40400): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.100545] softirqs last enabled at (39080): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.100548] softirqs last disabled at (39073): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.100550] ---[ end trace 0000000000000000 ]---
<7> [100.100583] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.100704] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<5> [100.100816] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4272, lrc_seqno=4272, guc_id=0, flags=0x73 in no process [-1]
<7> [100.100819] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.100878] ------------[ cut here ]------------
<4> [100.100879] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.100880] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#6: kworker/u64:4/204
<4> [100.100951] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.101008] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.101012] CPU: 6 UID: 0 PID: 204 Comm: kworker/u64:4 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.101015] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.101015] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.101017] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.101021] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.101089] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.101090] RSP: 0018:ffffc900014cbc98 EFLAGS: 00010246
<4> [100.101092] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.101093] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.101094] RBP: ffffc900014cbda8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.101095] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.101096] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.101097] FS: 0000000000000000(0000) GS:ffff8888dafda000(0000) knlGS:0000000000000000
<4> [100.101098] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.101100] CR2: 0000797c950e1078 CR3: 0000000114804003 CR4: 0000000000f72ef0
<4> [100.101101] PKRU: 55555554
<4> [100.101102] Call Trace:
<4> [100.101103] <TASK>
<4> [100.101107] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.101112] ? lock_release+0xce/0x280
<4> [100.101117] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.101122] process_one_work+0x22e/0x6b0
<4> [100.101127] worker_thread+0x1e8/0x3d0
<4> [100.101130] ? __pfx_worker_thread+0x10/0x10
<4> [100.101132] kthread+0x11f/0x250
<4> [100.101135] ? __pfx_kthread+0x10/0x10
<4> [100.101138] ret_from_fork+0x344/0x3a0
<4> [100.101140] ? __pfx_kthread+0x10/0x10
<4> [100.101143] ret_from_fork_asm+0x1a/0x30
<4> [100.101150] </TASK>
<4> [100.101151] irq event stamp: 41253
<4> [100.101151] hardirqs last enabled at (41259): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.101154] hardirqs last disabled at (41264): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.101156] softirqs last enabled at (39080): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.101158] softirqs last disabled at (39073): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.101161] ---[ end trace 0000000000000000 ]---
<6> [100.101162] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.101229] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.101235] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.101285] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.101556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.101644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.101741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.101828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.101911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.101993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.102075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.102153] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.102245] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.102356] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.103364] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.107755] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling AUX_TC2
<7> [100.113742] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.114002] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.114870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.114945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.115018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.115091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.115164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.115235] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.115306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.115376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.115447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.115518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.115589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.115659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.115729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.115818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.115888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.115959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.116030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.116099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.116169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.116251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.116330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.116405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.116481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.116555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.116626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.116700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.116788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.116864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.116939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.117014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.117088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.117168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.117246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.117324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.117401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.117475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.117547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.117621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.117692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.117777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.117851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.117925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.117996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.118077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.118154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.118228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.118303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.118378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.118450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.118574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.118667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.118749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.118824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.118896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.118971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.119042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.119113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.119187] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.119488] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.119493] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4272, lrc_seqno=4272, guc_id=0, flags=0x73 in no process [-1]
<7> [100.119495] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.119553] ------------[ cut here ]------------
<4> [100.119554] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.119555] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#0: kworker/u64:11/211
<4> [100.119623] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.119681] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.119686] CPU: 0 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.119689] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.119690] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.119691] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.119696] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.119779] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.119781] RSP: 0018:ffffc90001667c98 EFLAGS: 00010246
<4> [100.119783] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.119785] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.119786] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.119787] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.119788] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.119789] FS: 0000000000000000(0000) GS:ffff8888dacda000(0000) knlGS:0000000000000000
<4> [100.119791] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.119792] CR2: 0000797c95526778 CR3: 0000000003448005 CR4: 0000000000f72ef0
<4> [100.119793] PKRU: 55555554
<4> [100.119794] Call Trace:
<4> [100.119795] <TASK>
<4> [100.119800] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.119805] ? lock_release+0xce/0x280
<4> [100.119811] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.119816] process_one_work+0x22e/0x6b0
<4> [100.119822] worker_thread+0x1e8/0x3d0
<4> [100.119825] ? __pfx_worker_thread+0x10/0x10
<4> [100.119827] kthread+0x11f/0x250
<4> [100.119831] ? __pfx_kthread+0x10/0x10
<4> [100.119834] ret_from_fork+0x344/0x3a0
<4> [100.119836] ? __pfx_kthread+0x10/0x10
<4> [100.119839] ret_from_fork_asm+0x1a/0x30
<4> [100.119845] </TASK>
<4> [100.119846] irq event stamp: 158503
<4> [100.119847] hardirqs last enabled at (158509): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.119850] hardirqs last disabled at (158514): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.119852] softirqs last enabled at (158342): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.119855] softirqs last disabled at (158313): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.119857] ---[ end trace 0000000000000000 ]---
<6> [100.119859] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.119926] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.120728] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.120835] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.120945] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [100.121073] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.121343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.121430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.121518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.121600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.121682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.121774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.121856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.121935] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.122028] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.122140] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.123153] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.133738] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.133995] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.134911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.134989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.135063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.135136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.135210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.135281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.135352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.135422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.135493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.135565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.135634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.135704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.135797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.135868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.135937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.136007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.136077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.136148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.136218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.136296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.136375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.136451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.136527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.136602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.136676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.136759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.136839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.136917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.136994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.137071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.137147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.137228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.137309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.137387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.137464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.137539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.137611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.137683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.137763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.137837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.137910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.137982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.138054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.138130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.138202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.138277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.138352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.138428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.138499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.138571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.138646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.138720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.138807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.138879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.138956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.139030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.139102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.139178] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.139518] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.139522] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4272, lrc_seqno=4272, guc_id=0, flags=0x73 in no process [-1]
<7> [100.139524] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.139583] ------------[ cut here ]------------
<4> [100.139584] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.139585] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.139655] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.139713] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.139718] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.139720] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.139721] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.139723] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.139727] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.139813] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.139815] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.139817] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.139819] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.139820] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.139821] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.139822] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.139824] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.139825] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.139827] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.139828] PKRU: 55555554
<4> [100.139829] Call Trace:
<4> [100.139830] <TASK>
<4> [100.139836] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.139840] ? lock_release+0xce/0x280
<4> [100.139846] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.139851] process_one_work+0x22e/0x6b0
<4> [100.139856] worker_thread+0x1e8/0x3d0
<4> [100.139859] ? __pfx_worker_thread+0x10/0x10
<4> [100.139861] kthread+0x11f/0x250
<4> [100.139865] ? __pfx_kthread+0x10/0x10
<4> [100.139868] ret_from_fork+0x344/0x3a0
<4> [100.139870] ? __pfx_kthread+0x10/0x10
<4> [100.139873] ret_from_fork_asm+0x1a/0x30
<4> [100.139879] </TASK>
<4> [100.139880] irq event stamp: 163085
<4> [100.139881] hardirqs last enabled at (163091): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.139884] hardirqs last disabled at (163096): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.139886] softirqs last enabled at (161674): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.139889] softirqs last disabled at (161669): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.139891] ---[ end trace 0000000000000000 ]---
<7> [100.140549] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [100.140716] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4274, lrc_seqno=4274, guc_id=0, flags=0x73 in no process [-1]
<7> [100.140719] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.140817] ------------[ cut here ]------------
<4> [100.140819] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.140820] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.140903] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.140960] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.140964] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.140967] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.140968] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.140969] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.140973] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.141042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.141043] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.141045] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.141046] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.141048] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.141049] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.141049] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.141051] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.141052] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.141053] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.141054] PKRU: 55555554
<4> [100.141055] Call Trace:
<4> [100.141056] <TASK>
<4> [100.141061] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.141066] ? lock_release+0xce/0x280
<4> [100.141071] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.141076] process_one_work+0x22e/0x6b0
<4> [100.141081] worker_thread+0x1e8/0x3d0
<4> [100.141084] ? __pfx_worker_thread+0x10/0x10
<4> [100.141086] kthread+0x11f/0x250
<4> [100.141089] ? __pfx_kthread+0x10/0x10
<4> [100.141092] ret_from_fork+0x344/0x3a0
<4> [100.141094] ? __pfx_kthread+0x10/0x10
<4> [100.141097] ret_from_fork_asm+0x1a/0x30
<4> [100.141104] </TASK>
<4> [100.141105] irq event stamp: 163951
<4> [100.141106] hardirqs last enabled at (163957): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.141108] hardirqs last disabled at (163962): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.141110] softirqs last enabled at (163184): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.141113] softirqs last disabled at (163179): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.141115] ---[ end trace 0000000000000000 ]---
<6> [100.141116] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.141182] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.141188] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.141241] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.141339] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [100.141439] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.141704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.141799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.141889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.141973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.142056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.142138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.142218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.142297] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.142387] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.142499] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.143507] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.153740] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.153997] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.154834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.154909] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.154982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.155054] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.155126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.155197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.155266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.155336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.155406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.155476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.155547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.155617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.155686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.155763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.155834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.155904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.155973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.156044] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.156112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.156190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.156267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.156341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.156418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.156492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.156565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.156641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.156718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.156805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.156882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.156960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.157036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.157116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.157194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.157273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.157349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.157423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.157495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.157567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.157638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.157713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.157801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.157875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.157946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.158023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.158097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.158168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.158239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.158311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.158381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.158454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.158532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.158606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.158681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.158759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.158833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.158904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.158977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.159052] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.159371] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.160174] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4274, lrc_seqno=4274, guc_id=0, flags=0x73 in no process [-1]
<7> [100.160177] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.160236] ------------[ cut here ]------------
<4> [100.160237] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.160238] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.160307] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.160364] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.160369] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.160371] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.160372] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.160373] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.160378] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.160445] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.160447] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.160449] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.160450] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.160451] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.160452] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.160453] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.160455] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.160456] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.160457] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.160458] PKRU: 55555554
<4> [100.160459] Call Trace:
<4> [100.160460] <TASK>
<4> [100.160465] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.160470] ? lock_release+0xce/0x280
<4> [100.160475] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.160480] process_one_work+0x22e/0x6b0
<4> [100.160486] worker_thread+0x1e8/0x3d0
<4> [100.160489] ? __pfx_worker_thread+0x10/0x10
<4> [100.160491] kthread+0x11f/0x250
<4> [100.160494] ? __pfx_kthread+0x10/0x10
<4> [100.160497] ret_from_fork+0x344/0x3a0
<4> [100.160499] ? __pfx_kthread+0x10/0x10
<4> [100.160502] ret_from_fork_asm+0x1a/0x30
<4> [100.160509] </TASK>
<4> [100.160510] irq event stamp: 168461
<4> [100.160511] hardirqs last enabled at (168467): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.160513] hardirqs last disabled at (168472): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.160515] softirqs last enabled at (167154): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.160518] softirqs last disabled at (167149): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.160520] ---[ end trace 0000000000000000 ]---
<6> [100.160521] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.160590] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.160596] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.160647] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.160752] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [100.160852] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.161117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.161201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.161288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.161371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.161455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.161537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.161621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.161701] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.161805] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.161915] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.162922] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.172737] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.172989] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.173793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.173867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.173938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.174011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.174087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.174158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.174230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.174301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.174371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.174442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.174512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.174582] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.174653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.174724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.174808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.174879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.174948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.175018] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.175089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.175167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.175243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.175320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.175396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.175468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.175541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.175617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.175697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.175784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.175860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.175935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.176010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.176091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.176171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.176251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.176326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.176401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.176474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.176545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.176617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.176691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.176772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.176844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.176918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.176995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.177068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.177140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.177212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.177287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.177359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.177435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.177512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.177587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.177661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.177737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.177810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.177881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.177955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.178029] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.178367] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.178370] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4274, lrc_seqno=4274, guc_id=0, flags=0x73 in no process [-1]
<7> [100.178373] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.178430] ------------[ cut here ]------------
<4> [100.178431] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.178432] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.178501] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.178558] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.178562] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.178565] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.178566] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.178567] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.178571] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.178639] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.178640] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.178642] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.178643] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.178645] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.178646] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.178647] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.178648] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.178649] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.178650] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.178652] PKRU: 55555554
<4> [100.178652] Call Trace:
<4> [100.178653] <TASK>
<4> [100.178658] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.178663] ? lock_release+0xce/0x280
<4> [100.178668] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.178673] process_one_work+0x22e/0x6b0
<4> [100.178679] worker_thread+0x1e8/0x3d0
<4> [100.178681] ? __pfx_worker_thread+0x10/0x10
<4> [100.178684] kthread+0x11f/0x250
<4> [100.178687] ? __pfx_kthread+0x10/0x10
<4> [100.178690] ret_from_fork+0x344/0x3a0
<4> [100.178692] ? __pfx_kthread+0x10/0x10
<4> [100.178695] ret_from_fork_asm+0x1a/0x30
<4> [100.178701] </TASK>
<4> [100.178702] irq event stamp: 172975
<4> [100.178703] hardirqs last enabled at (172981): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.178706] hardirqs last disabled at (172986): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.178708] softirqs last enabled at (171472): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.178710] softirqs last disabled at (171463): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.178712] ---[ end trace 0000000000000000 ]---
<7> [100.179125] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [100.179163] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<5> [100.179556] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4277, lrc_seqno=4277, guc_id=0, flags=0x73 in no process [-1]
<7> [100.179559] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.179619] ------------[ cut here ]------------
<4> [100.179619] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.179621] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.179686] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.179749] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.179754] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.179757] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.179758] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.179759] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.179764] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.179830] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.179832] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.179834] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.179835] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.179836] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.179837] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.179838] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.179839] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.179840] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.179841] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.179843] PKRU: 55555554
<4> [100.179844] Call Trace:
<4> [100.179844] <TASK>
<4> [100.179849] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.179853] ? lock_release+0xce/0x280
<4> [100.179859] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.179864] process_one_work+0x22e/0x6b0
<4> [100.179869] worker_thread+0x1e8/0x3d0
<4> [100.179872] ? __pfx_worker_thread+0x10/0x10
<4> [100.179874] kthread+0x11f/0x250
<4> [100.179877] ? __pfx_kthread+0x10/0x10
<4> [100.179880] ret_from_fork+0x344/0x3a0
<4> [100.179882] ? __pfx_kthread+0x10/0x10
<4> [100.179885] ret_from_fork_asm+0x1a/0x30
<4> [100.179891] </TASK>
<4> [100.179892] irq event stamp: 173843
<4> [100.179893] hardirqs last enabled at (173849): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.179895] hardirqs last disabled at (173854): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.179897] softirqs last enabled at (171472): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.179900] softirqs last disabled at (171463): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.179902] ---[ end trace 0000000000000000 ]---
<6> [100.179903] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.179968] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.179974] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.180030] xe 0000:03:00.0: [drm:drm_pagemap_migrate_to_devmem [drm_gpusvm_helper]] Total pages 1; Own pages: 0.
<7> [100.180030] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.180127] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [100.180228] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.180490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.180577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.180666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.180755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.180837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.180918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.181000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.181078] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.181168] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.181277] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.182287] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.192737] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
<7> [100.192992] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled
<7> [100.193827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1
<7> [100.193902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16
<7> [100.193975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc
<7> [100.194049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c
<7> [100.194121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130
<7> [100.194194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c
<7> [100.194264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100
<7> [100.194335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100
<7> [100.194407] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100
<7> [100.194477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100
<7> [100.194545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100
<7> [100.194614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100
<7> [100.194683] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100
<7> [100.194761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100
<7> [100.194830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100
<7> [100.194899] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100
<7> [100.194967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100
<7> [100.195038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100
<7> [100.195107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs
<7> [100.195184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000
<7> [100.195262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306
<7> [100.195338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080
<7> [100.195413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x1000dafc
<7> [100.195486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x1000db01
<7> [100.195559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x0000db1c
<7> [100.195635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x7044] = 0x04200420
<7> [100.195715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040
<7> [100.195801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200
<7> [100.195874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000
<7> [100.195951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040
<7> [100.196029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002
<7> [100.196110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400
<7> [100.196189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000
<7> [100.196266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009000
<7> [100.196342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002
<7> [100.196418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs
<7> [100.196492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000
<7> [100.196565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306
<7> [100.196636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080
<7> [100.196712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs
<7> [100.196795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000
<7> [100.196867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306
<7> [100.196939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080
<7> [100.197015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs
<7> [100.197087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000
<7> [100.197159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308
<7> [100.197230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080
<7> [100.197304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc
<7> [100.197375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01
<7> [100.197445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c
<7> [100.197519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs
<7> [100.197591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000
<7> [100.197663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308
<7> [100.197736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080
<7> [100.197809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc
<7> [100.197882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01
<7> [100.197956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c
<7> [100.198032] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2
<6> [100.198344] xe 0000:03:00.0: [drm] Tile0: GT0: reset done
<5> [100.198348] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4277, lrc_seqno=4277, guc_id=0, flags=0x73 in no process [-1]
<7> [100.198350] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4> [100.198406] ------------[ cut here ]------------
<4> [100.198407] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4> [100.198409] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x13f8/0x23e0 [xe], CPU#4: kworker/u64:11/211
<4> [100.198478] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp cmdlinepart coretemp hid_generic eeepc_wmi spi_nor asus_wmi mei_hdcp mtd mei_pxp sparse_keymap platform_profile wmi_bmof kvm_intel usbhid kvm hid snd_hda_intel irqbypass snd_intel_dspcfg ghash_clmulni_intel r8169 aesni_intel snd_hda_codec rapl snd_hda_core video snd_hwdep intel_cstate snd_pcm binfmt_misc realtek snd_timer i2c_i801 idma64 i2c_mux mei_me snd spi_intel_pci spi_intel soundcore i2c_smbus intel_pmc_core mei pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry pinctrl_alderlake intel_vsec wmi acpi_pad acpi_tad dm_multipath msr nvme_fabrics efi_pstore
<4> [100.198534] fuse nfnetlink autofs4 [last unloaded: xe_live_test]
<4> [100.198539] CPU: 4 UID: 0 PID: 211 Comm: kworker/u64:11 Tainted: G S U W N 6.19.0-lgci-xe-xe-4532-d2a98b98376b6a16c+ #1 PREEMPT(voluntary)
<4> [100.198541] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [N]=TEST
<4> [100.198542] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4> [100.198544] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4> [100.198548] RIP: 0010:guc_exec_queue_timedout_job+0x1401/0x23e0 [xe]
<4> [100.198614] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 62 3b 5a e1 48 89 c6 48 8d 3d d8 af 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 f2 ee ff ff 8b 70 08 49
<4> [100.198616] RSP: 0000:ffffc90001667c98 EFLAGS: 00010246
<4> [100.198618] RAX: ffffffffa11f9cb4 RBX: 0000000000000000 RCX: 0000000000000000
<4> [100.198619] RDX: ffff8881046a6f10 RSI: ffffffffa11f9cb4 RDI: ffffffffa1003e00
<4> [100.198620] RBP: ffffc90001667da8 R08: 0000000000000000 R09: 0000000000000000
<4> [100.198621] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4> [100.198622] R13: ffff8881046a6f10 R14: ffff888121f30818 R15: 00000000ffffffc2
<4> [100.198624] FS: 0000000000000000(0000) GS:ffff8888daeda000(0000) knlGS:0000000000000000
<4> [100.198625] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4> [100.198626] CR2: 0000797c95542778 CR3: 0000000003448002 CR4: 0000000000f72ef0
<4> [100.198627] PKRU: 55555554
<4> [100.198628] Call Trace:
<4> [100.198629] <TASK>
<4> [100.198634] ? drm_sched_job_timedout+0x80/0x1a0 [gpu_sched]
<4> [100.198639] ? lock_release+0xce/0x280
<4> [100.198644] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4> [100.198649] process_one_work+0x22e/0x6b0
<4> [100.198655] worker_thread+0x1e8/0x3d0
<4> [100.198657] ? __pfx_worker_thread+0x10/0x10
<4> [100.198659] kthread+0x11f/0x250
<4> [100.198663] ? __pfx_kthread+0x10/0x10
<4> [100.198666] ret_from_fork+0x344/0x3a0
<4> [100.198668] ? __pfx_kthread+0x10/0x10
<4> [100.198671] ret_from_fork_asm+0x1a/0x30
<4> [100.198677] </TASK>
<4> [100.198678] irq event stamp: 178333
<4> [100.198679] hardirqs last enabled at (178339): [<ffffffff8149e939>] __up_console_sem+0x79/0xa0
<4> [100.198681] hardirqs last disabled at (178344): [<ffffffff8149e91e>] __up_console_sem+0x5e/0xa0
<4> [100.198683] softirqs last enabled at (176830): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.198686] softirqs last disabled at (176823): [<ffffffff813c968f>] __irq_exit_rcu+0x13f/0x160
<4> [100.198688] ---[ end trace 0000000000000000 ]---
<6> [100.198689] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<6> [100.198764] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued
<6> [100.199565] xe 0000:03:00.0: [drm] Tile0: GT0: reset started
<7> [100.199693] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations
<7> [100.199799] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed
<7> [100.199899] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped
<7> [100.200162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs
<7> [100.200244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000
<7> [100.200330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000
<7> [100.200411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x55555440
<7> [100.200493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440
<7> [100.200571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000
<7> [100.200650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f
<7> [100.200731] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K
<7> [100.200821] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K)
<7> [100.200931] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled
<7> [100.201935] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152)
<7> [100.211736] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034
Created at 2026-02-10 05:18:40