igt@xe_pat@pt-caching-update-pat-and-pte on shard-bmg@IGT

Detail	Value
Duration	unknown
Hostname	shard-bmg-9
Igt-Version	IGT-Version: 2.4-g85c00333c (x86_64) (Linux: 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ x86_64)
Out	Using IGT_SRANDOM=1776853151 for randomisation Opened device: /dev/dri/card0 Test requirement not met in function __igt_unique____real_main2210, file ../tests/intel/xe_pat.c:2298: Test requirement: intel_graphics_ver(dev_id) == IP_VER(35, 10) Last errno: 2, No such file or directory Starting subtest: pt-caching-update-pat-and-pte va_bits: 48, num_objs: 16384, spread over pt levels: [0]: 16384 [1]: 512 [2]: 16 runner: This test was killed due to a kernel taint (0x244). This test caused an abort condition: Kernel badly tainted (0x244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened.
Err	Starting subtest: pt-caching-update-pat-and-pte Received signal SIGQUIT. Stack trace: #0 [fatal_sig_handler+0x17b] #1 [__sigaction+0x50] #2 [ioctl+0x3d] #3 [drmIoctl+0x30] #4 [syncobj_create+0x46] #5 [pt_bind_objects+0x8e] #6 [pt_caching_test+0x302] #7 [__igt_unique____real_main2210+0x1443] #8 [main+0x3a] #9 [__libc_init_first+0x8a] #10 [__libc_start_main+0x8b] #11 [_start+0x25]
Dmesg Scroll to first warning	<6> [295.951449] Console: switching to colour dummy device 80x25 <6> [295.951772] [IGT] xe_pat: executing <6> [295.975331] [IGT] xe_pat: starting subtest pt-caching-update-pat-and-pte <7> [296.705523] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling AUX_TC2 <7> [296.809502] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling AUX_TC2 <6> [297.181616] xe 0000:03:00.0: [drm] Tile0: GT0: ASID: 0 Faulted Address: 0x0000780418800000 FaultType: 0 AccessType: 0 FaultLevel: 3 EngineClass: 3 bcs EngineInstance: 8 <6> [297.181627] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EINVAL <6> [297.181672] xe 0000:03:00.0: [drm] Tile0: GT0: Engine memory CAT error [18]: class=bcs, logical_mask: 0x2, guc_id=0 <7> [297.182365] xe 0000:03:00.0: [drm:xe_hw_engine_snapshot_capture [xe]] Tile0: GT0: Proceeding with manual engine snapshot <6> [297.182867] xe 0000:03:00.0: [drm] Tile0: GT0: Engine reset: engine_class=bcs, logical_mask: 0x2, guc_id=0, state=0x249 <5> [297.182877] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39946, lrc_seqno=39946, guc_id=0, flags=0x73 in no process [-1] <6> [297.362720] xe 0000:03:00.0: [drm] Xe device coredump has been created <6> [297.362741] xe 0000:03:00.0: [drm] Check your /sys/class/drm/card0/device/devcoredump/data <4> [297.362744] ------------[ cut here ]------------ <4> [297.362746] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.362747] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:3/193 <4> [297.362858] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.362925] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.362933] CPU: 8 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.362937] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER <4> [297.362938] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.362939] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.362945] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.363017] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.363018] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.363021] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.363022] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.363023] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.363024] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.363026] R13: ffff88810396d490 R14: ffff888114800000 R15: 00000000fffffffb <4> [297.363027] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [297.363028] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.363029] CR2: 000077f18c346afc CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.363031] PKRU: 55555554 <4> [297.363032] Call Trace: <4> [297.363033] <TASK> <4> [297.363037] ? lock_sync+0x100/0x100 <4> [297.363043] ? __pfx_autoremove_wake_function+0x10/0x10 <4> [297.363048] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.363053] process_one_work+0x239/0x760 <4> [297.363060] worker_thread+0x200/0x3f0 <4> [297.363062] ? __pfx_worker_thread+0x10/0x10 <4> [297.363065] kthread+0x10d/0x150 <4> [297.363068] ? __pfx_kthread+0x10/0x10 <4> [297.363071] ret_from_fork+0x3d4/0x480 <4> [297.363074] ? __pfx_kthread+0x10/0x10 <4> [297.363077] ret_from_fork_asm+0x1a/0x30 <4> [297.363084] </TASK> <4> [297.363085] irq event stamp: 1810511 <4> [297.363087] hardirqs last enabled at (1810517): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.363090] hardirqs last disabled at (1810522): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.363092] softirqs last enabled at (1809586): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.363095] softirqs last disabled at (1809581): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.363097] ---[ end trace 0000000000000000 ]--- <6> [297.363099] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.363170] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.363368] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.363422] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.364140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.364233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.364321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.364406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.364500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.364588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.364678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.364761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.364837] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.364923] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.365030] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.366159] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.376852] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.377128] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.378996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.379076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.379149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.379221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.379291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.379361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.379430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.379520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.379596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.379670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.379745] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.379820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.379894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.379972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.380050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.380126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.380202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.380277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.380352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.380434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.380606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.380686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.380767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.380848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.380927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.381003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.381079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.381158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.381239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.381316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.381394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.381553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.381643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.381730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.381816] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.381899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.381980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.382057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.382134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.382210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.382288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.382366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.382447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.382528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.382612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.382692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.382770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.382848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.382927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.383004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.383081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.383162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.383244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.383326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.383406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.383493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.383572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.383650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.383730] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.384241] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.384248] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39946, lrc_seqno=39946, guc_id=0, flags=0x73 in no process [-1] <7> [297.384251] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.384313] ------------[ cut here ]------------ <4> [297.384314] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.384316] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#10: kworker/u64:3/193 <4> [297.384391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.384497] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.384507] CPU: 10 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.384510] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.384511] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.384513] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.384518] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.384592] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.384594] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.384597] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.384598] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.384599] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.384601] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.384602] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.384603] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [297.384605] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.384606] CR2: 00007d61d560b048 CR3: 0000000115783004 CR4: 0000000000f72ef0 <4> [297.384608] PKRU: 55555554 <4> [297.384609] Call Trace: <4> [297.384610] <TASK> <4> [297.384614] ? lock_sync+0x100/0x100 <4> [297.384620] ? lock_release+0xd0/0x2b0 <4> [297.384626] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.384631] process_one_work+0x239/0x760 <4> [297.384638] worker_thread+0x200/0x3f0 <4> [297.384641] ? __pfx_worker_thread+0x10/0x10 <4> [297.384644] kthread+0x10d/0x150 <4> [297.384647] ? __pfx_kthread+0x10/0x10 <4> [297.384650] ret_from_fork+0x3d4/0x480 <4> [297.384653] ? __pfx_kthread+0x10/0x10 <4> [297.384656] ret_from_fork_asm+0x1a/0x30 <4> [297.384664] </TASK> <4> [297.384665] irq event stamp: 1813975 <4> [297.384667] hardirqs last enabled at (1813981): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.384670] hardirqs last disabled at (1813986): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.384672] softirqs last enabled at (1813532): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.384675] softirqs last disabled at (1813517): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.384677] ---[ end trace 0000000000000000 ]--- <6> [297.384679] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.384752] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.384758] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.385166] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.385460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.385560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.385655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.385745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.385832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.385917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.386003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.386090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.386171] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.386266] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.386383] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.387397] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.397447] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.397700] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.398816] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.398952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.399075] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.399198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.399319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.399454] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.399575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.399696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.399817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.399937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.400058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.400179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.400299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.400420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.400547] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.400668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.400788] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.400908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.401029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.401155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.401281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.401405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.401536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.401660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.401783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.401905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.402028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.402155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.402282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.402407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.402538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.402665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.402797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.402927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.403059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.403187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.403315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.403445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.403570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.403694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.403822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.403947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.404072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.404196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.404324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.404453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.404578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.404701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.404827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.404949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.405072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.405199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.405324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.405456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.405580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.405705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.405828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.405952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.406079] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.406413] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.406421] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=39946, lrc_seqno=39946, guc_id=0, flags=0x73 in no process [-1] <7> [297.406425] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.406537] ------------[ cut here ]------------ <4> [297.406539] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.406542] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:3/193 <4> [297.406655] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.406750] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.406762] CPU: 12 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.406766] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.406767] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.406770] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.406777] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.406889] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.406891] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.406894] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.406896] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.406898] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.406900] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.406902] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.406904] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [297.406906] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.406908] CR2: 0000639d59095650 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.406910] PKRU: 55555554 <4> [297.406912] Call Trace: <4> [297.406914] <TASK> <4> [297.406920] ? lock_sync+0x100/0x100 <4> [297.406928] ? lock_release+0xd0/0x2b0 <4> [297.406937] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.406946] process_one_work+0x239/0x760 <4> [297.406956] worker_thread+0x200/0x3f0 <4> [297.406960] ? __pfx_worker_thread+0x10/0x10 <4> [297.406964] kthread+0x10d/0x150 <4> [297.406968] ? __pfx_kthread+0x10/0x10 <4> [297.406973] ret_from_fork+0x3d4/0x480 <4> [297.406976] ? __pfx_kthread+0x10/0x10 <4> [297.406981] ret_from_fork_asm+0x1a/0x30 <4> [297.406994] </TASK> <4> [297.406995] irq event stamp: 1817089 <4> [297.406997] hardirqs last enabled at (1817095): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.407001] hardirqs last disabled at (1817100): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.407004] softirqs last enabled at (1816298): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.407008] softirqs last disabled at (1816293): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.407011] ---[ end trace 0000000000000000 ]--- <5> [297.409747] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40001, lrc_seqno=40001, guc_id=0, flags=0x73 in no process [-1] <7> [297.409753] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.409864] ------------[ cut here ]------------ <4> [297.409866] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.409868] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:3/193 <4> [297.409982] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.410072] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.410084] CPU: 12 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.410087] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.410089] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.410091] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.410099] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.410211] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.410213] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.410216] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.410218] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.410220] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.410221] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.410223] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.410225] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [297.410227] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.410229] CR2: 0000639d59095650 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.410231] PKRU: 55555554 <4> [297.410233] Call Trace: <4> [297.410234] <TASK> <4> [297.410241] ? lock_sync+0x100/0x100 <4> [297.410247] ? lock_release+0xd0/0x2b0 <4> [297.410256] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.410265] process_one_work+0x239/0x760 <4> [297.410274] worker_thread+0x200/0x3f0 <4> [297.410278] ? __pfx_worker_thread+0x10/0x10 <4> [297.410282] kthread+0x10d/0x150 <4> [297.410286] ? __pfx_kthread+0x10/0x10 <4> [297.410291] ret_from_fork+0x3d4/0x480 <4> [297.410294] ? __pfx_kthread+0x10/0x10 <4> [297.410298] ret_from_fork_asm+0x1a/0x30 <4> [297.410311] </TASK> <4> [297.410312] irq event stamp: 1818297 <4> [297.410314] hardirqs last enabled at (1818303): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.410317] hardirqs last disabled at (1818308): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.410321] softirqs last enabled at (1816298): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.410324] softirqs last disabled at (1816293): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.410327] ---[ end trace 0000000000000000 ]--- <6> [297.410329] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.410448] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.410458] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.410978] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.411137] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.411807] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.412065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.412197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.412335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.412475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.412607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.412738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.412871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.413005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.413135] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.413278] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.413449] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.414790] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.425450] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.425771] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.427025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.427150] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.427270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.427391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.427531] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.427621] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.427705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.427783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.427862] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.427940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.428017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.428095] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.428175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.428253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.428328] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.428404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.428491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.428567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.428641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.428723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.428804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.428884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.428966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.429048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.429128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.429207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.429284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.429364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.429451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.429529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.429608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.429686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.429771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.429855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.429939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.430025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.430108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.430187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.430266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.430343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.430424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.430514] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.430593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.430673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.430756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.430835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.430913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.430990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.431069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.431145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.431221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.431301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.431378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.431461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.431538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.431619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.431699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.431781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.431864] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.431979] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.431984] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40001, lrc_seqno=40001, guc_id=0, flags=0x73 in no process [-1] <7> [297.431986] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.432046] ------------[ cut here ]------------ <4> [297.432047] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.432049] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:3/193 <4> [297.432120] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.432186] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.432194] CPU: 4 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.432197] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.432199] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.432200] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.432205] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.432274] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.432276] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.432279] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.432280] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.432281] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.432283] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.432284] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.432285] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.432287] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.432288] CR2: 000072fba006c418 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.432290] PKRU: 55555554 <4> [297.432291] Call Trace: <4> [297.432292] <TASK> <4> [297.432296] ? lock_sync+0x100/0x100 <4> [297.432300] ? lock_release+0xd0/0x2b0 <4> [297.432306] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.432312] process_one_work+0x239/0x760 <4> [297.432319] worker_thread+0x200/0x3f0 <4> [297.432321] ? __pfx_worker_thread+0x10/0x10 <4> [297.432324] kthread+0x10d/0x150 <4> [297.432327] ? __pfx_kthread+0x10/0x10 <4> [297.432330] ret_from_fork+0x3d4/0x480 <4> [297.432332] ? __pfx_kthread+0x10/0x10 <4> [297.432336] ret_from_fork_asm+0x1a/0x30 <4> [297.432343] </TASK> <4> [297.432344] irq event stamp: 1821401 <4> [297.432345] hardirqs last enabled at (1821407): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.432348] hardirqs last disabled at (1821412): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.432351] softirqs last enabled at (1820498): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.432353] softirqs last disabled at (1820493): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.432355] ---[ end trace 0000000000000000 ]--- <6> [297.432357] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.432427] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.432433] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.432845] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.433049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.433137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.433231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.433318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.433404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.433500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.433587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.433673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.433755] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.433852] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.433971] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.434980] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.445442] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.445690] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.447010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.447089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.447166] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.447244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.447322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.447399] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.447489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.447568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.447644] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.447720] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.447795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.447870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.447944] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.448019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.448094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.448169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.448244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.448322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.448399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.448490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.448573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.448653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.448734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.448813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.448890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.448968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.449046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.449126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.449208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.449292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.449374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.449459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.449544] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.449630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.449714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.449801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.449884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.449963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.450041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.450118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.450198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.450276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.450353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.450429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.450523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.450601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.450682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.450763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.450843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.450920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.450996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.451075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.451154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.451234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.451314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.451395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.451482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.451559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.451638] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.451751] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.451755] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40001, lrc_seqno=40001, guc_id=0, flags=0x73 in no process [-1] <7> [297.451757] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.451816] ------------[ cut here ]------------ <4> [297.451817] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.451819] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:3/193 <4> [297.451891] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.451955] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.451963] CPU: 4 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.451966] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.451967] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.451968] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.451973] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.452042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.452044] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.452046] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.452048] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.452049] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.452050] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.452052] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.452053] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.452055] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.452056] CR2: 000072fba006c418 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.452057] PKRU: 55555554 <4> [297.452059] Call Trace: <4> [297.452060] <TASK> <4> [297.452063] ? lock_sync+0x100/0x100 <4> [297.452068] ? lock_release+0xd0/0x2b0 <4> [297.452073] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.452079] process_one_work+0x239/0x760 <4> [297.452085] worker_thread+0x200/0x3f0 <4> [297.452088] ? __pfx_worker_thread+0x10/0x10 <4> [297.452090] kthread+0x10d/0x150 <4> [297.452093] ? __pfx_kthread+0x10/0x10 <4> [297.452097] ret_from_fork+0x3d4/0x480 <4> [297.452099] ? __pfx_kthread+0x10/0x10 <4> [297.452102] ret_from_fork_asm+0x1a/0x30 <4> [297.452109] </TASK> <4> [297.452111] irq event stamp: 1824491 <4> [297.452112] hardirqs last enabled at (1824497): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.452114] hardirqs last disabled at (1824502): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.452117] softirqs last enabled at (1823606): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.452119] softirqs last disabled at (1823595): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.452121] ---[ end trace 0000000000000000 ]--- <5> [297.453404] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40056, lrc_seqno=40056, guc_id=0, flags=0x73 in no process [-1] <7> [297.453411] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [297.453458] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [297.453511] ------------[ cut here ]------------ <4> [297.453513] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.453514] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.453590] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr <7> [297.453564] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [297.453612] intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.453665] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.453668] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.453669] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.453670] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.453676] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.453748] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.453749] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.453752] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.453753] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.453755] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.453756] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.453757] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.453759] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.453760] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.453762] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.453763] PKRU: 55555554 <4> [297.453764] Call Trace: <4> [297.453765] <TASK> <4> [297.453770] ? lock_sync+0x100/0x100 <4> [297.453775] ? lock_release+0xd0/0x2b0 <4> [297.453780] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.453786] process_one_work+0x239/0x760 <4> [297.453792] worker_thread+0x200/0x3f0 <4> [297.453795] ? __pfx_worker_thread+0x10/0x10 <4> [297.453797] kthread+0x10d/0x150 <4> [297.453800] ? __pfx_kthread+0x10/0x10 <4> [297.453804] ret_from_fork+0x3d4/0x480 <4> [297.453806] ? __pfx_kthread+0x10/0x10 <4> [297.453809] ret_from_fork_asm+0x1a/0x30 <4> [297.453817] </TASK> <4> [297.453818] irq event stamp: 964497 <4> [297.453819] hardirqs last enabled at (964503): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.453822] hardirqs last disabled at (964508): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.453824] softirqs last enabled at (963706): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.453827] softirqs last disabled at (963699): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.453829] ---[ end trace 0000000000000000 ]--- <6> [297.453831] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.453903] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.453909] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.454038] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.454242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.454331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.454424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.454527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.454617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.454702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.454792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.454884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.454969] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.455064] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.455182] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.456188] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.466442] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.466691] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.467691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.467771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.467848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.467926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.468003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.468079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.468155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.468231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.468305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.468380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.468467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.468541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.468617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.468696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.468773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.468847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.468922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.468996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.469070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.469151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.469232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.469311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.469391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.469481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.469559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.469636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.469713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.469794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.469876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.469959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.470041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.470122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.470208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.470293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.470377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.470464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.470545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.470623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.470704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.470783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.470864] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.470944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.471023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.471099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.471179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.471256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.471333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.471410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.471496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.471572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.471648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.471726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.471802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.471878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.471954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.472030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.472105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.472180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.472262] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.472374] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.472378] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40056, lrc_seqno=40056, guc_id=0, flags=0x73 in no process [-1] <7> [297.472380] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.473581] ------------[ cut here ]------------ <4> [297.473582] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.473583] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:3/193 <4> [297.473660] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.473724] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.473732] CPU: 4 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.473735] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.473736] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.473738] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.473743] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.473812] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.473814] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.473816] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.473817] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.473819] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.473820] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.473821] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.473823] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.473824] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.473825] CR2: 000072fba006c418 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.473827] PKRU: 55555554 <4> [297.473828] Call Trace: <4> [297.473829] <TASK> <4> [297.473833] ? lock_sync+0x100/0x100 <4> [297.473838] ? lock_release+0xd0/0x2b0 <4> [297.473843] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.473849] process_one_work+0x239/0x760 <4> [297.473855] worker_thread+0x200/0x3f0 <4> [297.473858] ? __pfx_worker_thread+0x10/0x10 <4> [297.473860] kthread+0x10d/0x150 <4> [297.473863] ? __pfx_kthread+0x10/0x10 <4> [297.473867] ret_from_fork+0x3d4/0x480 <4> [297.473869] ? __pfx_kthread+0x10/0x10 <4> [297.473872] ret_from_fork_asm+0x1a/0x30 <4> [297.473880] </TASK> <4> [297.473881] irq event stamp: 1828725 <4> [297.473882] hardirqs last enabled at (1828731): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.473885] hardirqs last disabled at (1828736): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.473887] softirqs last enabled at (1827594): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.473889] softirqs last disabled at (1827585): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.473892] ---[ end trace 0000000000000000 ]--- <6> [297.473893] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.473963] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.473969] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.473985] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.474090] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.474380] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.474709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.474805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.474901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.474990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.475078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.475164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.475251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.475340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.475422] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.475532] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.475650] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.476672] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.486441] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.486690] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.487900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.487980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.488058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.488137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.488214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.488291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.488366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.488444] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.488520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.488594] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.488668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.488743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.488817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.488891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.488968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.489046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.489122] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.489197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.489272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.489354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.489439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.489518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.489599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.489676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.489754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.489835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.489915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.489997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.490077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.490156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.490236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.490315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.490399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.490492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.490577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.490657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.490736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.490813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.490890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.490965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.491044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.491121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.491197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.491274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.491354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.491437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.491519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.491598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.491677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.491755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.491831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.491910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.491987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.492064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.492139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.492216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.492290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.492366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.492451] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.492565] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.492569] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40056, lrc_seqno=40056, guc_id=0, flags=0x73 in no process [-1] <7> [297.492572] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.492630] ------------[ cut here ]------------ <4> [297.492632] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.492633] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:1/120 <4> [297.492705] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.492770] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.492779] CPU: 8 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.492782] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.492783] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.492784] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.492790] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.492860] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.492861] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.492864] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.492865] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.492866] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.492868] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.492869] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.492870] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [297.492872] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.492873] CR2: 000077f18c346afc CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.492875] PKRU: 55555554 <4> [297.492876] Call Trace: <4> [297.492877] <TASK> <4> [297.492881] ? lock_sync+0x100/0x100 <4> [297.492885] ? lock_release+0xd0/0x2b0 <4> [297.492891] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.492896] process_one_work+0x239/0x760 <4> [297.492903] worker_thread+0x200/0x3f0 <4> [297.492906] ? __pfx_worker_thread+0x10/0x10 <4> [297.492908] kthread+0x10d/0x150 <4> [297.492911] ? __pfx_kthread+0x10/0x10 <4> [297.492915] ret_from_fork+0x3d4/0x480 <4> [297.492917] ? __pfx_kthread+0x10/0x10 <4> [297.492920] ret_from_fork_asm+0x1a/0x30 <4> [297.492928] </TASK> <4> [297.492929] irq event stamp: 1011387 <4> [297.492930] hardirqs last enabled at (1011393): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.492933] hardirqs last disabled at (1011398): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.492935] softirqs last enabled at (1010514): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.492937] softirqs last disabled at (1010507): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.492940] ---[ end trace 0000000000000000 ]--- <5> [297.493293] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40111, lrc_seqno=40111, guc_id=0, flags=0x73 in no process [-1] <7> [297.493296] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.493354] ------------[ cut here ]------------ <4> [297.493355] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.493356] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:1/120 <4> [297.493426] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.493493] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.493501] CPU: 8 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.493504] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.493505] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.493506] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.493511] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.493579] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.493581] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.493583] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.493584] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.493585] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.493587] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.493588] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.493589] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [297.493591] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.493592] CR2: 000077f18c346afc CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.493593] PKRU: 55555554 <4> [297.493594] Call Trace: <4> [297.493595] <TASK> <4> [297.493599] ? lock_sync+0x100/0x100 <4> [297.493603] ? lock_release+0xd0/0x2b0 <4> [297.493608] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.493614] process_one_work+0x239/0x760 <4> [297.493620] worker_thread+0x200/0x3f0 <4> [297.493622] ? __pfx_worker_thread+0x10/0x10 <4> [297.493625] kthread+0x10d/0x150 <4> [297.493627] ? __pfx_kthread+0x10/0x10 <4> [297.493631] ret_from_fork+0x3d4/0x480 <4> [297.493633] ? __pfx_kthread+0x10/0x10 <4> [297.493636] ret_from_fork_asm+0x1a/0x30 <4> [297.493643] </TASK> <4> [297.493644] irq event stamp: 1013297 <4> [297.493646] hardirqs last enabled at (1013303): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.493648] hardirqs last disabled at (1013308): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.493650] softirqs last enabled at (1010514): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.493653] softirqs last disabled at (1010507): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.493655] ---[ end trace 0000000000000000 ]--- <6> [297.494574] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.494646] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.494654] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.494811] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.494917] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.495024] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.495231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.495323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.495418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.495521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.495607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.495694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.495780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.495867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.495949] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.496045] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.496163] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.497173] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.507441] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.507691] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.508724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.508803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.508878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.508957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.509035] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.509114] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.509193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.509270] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.509346] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.509422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.509510] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.509584] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.509660] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.509734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.509808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.509882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.509955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.510029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.510103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.510185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.510268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.510349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.510437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.510519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.510596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.510674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.510752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.510832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.510912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.510991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.511069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.511147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.511232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.511315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.511399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.511493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.511576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.511655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.511733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.511811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.511892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.511970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.512048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.512124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.512205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.512282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.512360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.512440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.512518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.512593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.512669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.512749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.512827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.512903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.512978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.513055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.513131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.513205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.513283] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.513395] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.513400] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40111, lrc_seqno=40111, guc_id=0, flags=0x73 in no process [-1] <7> [297.513402] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.514314] ------------[ cut here ]------------ <4> [297.514315] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.514317] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.514391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.514461] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.514470] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.514473] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.514474] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.514476] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.514481] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.514551] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.514553] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.514555] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.514556] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.514558] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.514559] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.514560] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.514561] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.514563] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.514564] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.514566] PKRU: 55555554 <4> [297.514567] Call Trace: <4> [297.514568] <TASK> <4> [297.514572] ? lock_sync+0x100/0x100 <4> [297.514577] ? lock_release+0xd0/0x2b0 <4> [297.514582] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.514588] process_one_work+0x239/0x760 <4> [297.514594] worker_thread+0x200/0x3f0 <4> [297.514597] ? __pfx_worker_thread+0x10/0x10 <4> [297.514599] kthread+0x10d/0x150 <4> [297.514602] ? __pfx_kthread+0x10/0x10 <4> [297.514606] ret_from_fork+0x3d4/0x480 <4> [297.514608] ? __pfx_kthread+0x10/0x10 <4> [297.514612] ret_from_fork_asm+0x1a/0x30 <4> [297.514619] </TASK> <4> [297.514620] irq event stamp: 969537 <4> [297.514621] hardirqs last enabled at (969543): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.514624] hardirqs last disabled at (969548): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.514626] softirqs last enabled at (968586): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.514629] softirqs last disabled at (968579): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.514631] ---[ end trace 0000000000000000 ]--- <7> [297.514645] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.514749] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [297.514846] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.514918] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.514924] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.515328] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.515533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.515622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.515715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.515805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.515893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.515981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.516071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.516161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.516243] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.516339] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.516459] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.517465] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.527440] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.527689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.528883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.528961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.529037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.529117] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.529196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.529273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.529348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.529423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.529511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.529585] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.529658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.529733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.529807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.529882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.529955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.530029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.530104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.530177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.530251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.530332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.530414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.530508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.530594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.530675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.530754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.530833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.530911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.530991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.531071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.531149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.531227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.531306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.531391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.531483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.531568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.531648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.531730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.531812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.531892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.531970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.532050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.532129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.532206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.532282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.532362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.532440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.532517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.532597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.532679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.532757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.532834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.532912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.532991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.533067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.533143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.533220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.533296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.533373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.533460] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.533572] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.533576] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40111, lrc_seqno=40111, guc_id=0, flags=0x73 in no process [-1] <7> [297.533578] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.533637] ------------[ cut here ]------------ <4> [297.533639] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.533640] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.533711] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.533776] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.533784] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.533787] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.533788] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.533789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.533795] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.533863] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.533865] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.533867] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.533868] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.533870] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.533871] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.533872] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.533873] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.533875] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.533876] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.533878] PKRU: 55555554 <4> [297.533879] Call Trace: <4> [297.533880] <TASK> <4> [297.533883] ? lock_sync+0x100/0x100 <4> [297.533888] ? lock_release+0xd0/0x2b0 <4> [297.533893] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.533899] process_one_work+0x239/0x760 <4> [297.533905] worker_thread+0x200/0x3f0 <4> [297.533908] ? __pfx_worker_thread+0x10/0x10 <4> [297.533910] kthread+0x10d/0x150 <4> [297.533913] ? __pfx_kthread+0x10/0x10 <4> [297.533917] ret_from_fork+0x3d4/0x480 <4> [297.533919] ? __pfx_kthread+0x10/0x10 <4> [297.533922] ret_from_fork_asm+0x1a/0x30 <4> [297.533929] </TASK> <4> [297.533931] irq event stamp: 972625 <4> [297.533932] hardirqs last enabled at (972631): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.533934] hardirqs last disabled at (972636): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.533937] softirqs last enabled at (971572): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.533939] softirqs last disabled at (971565): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.533941] ---[ end trace 0000000000000000 ]--- <5> [297.535231] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40166, lrc_seqno=40166, guc_id=0, flags=0x73 in no process [-1] <7> [297.535235] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.535300] ------------[ cut here ]------------ <4> [297.535301] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.535302] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.535375] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.535443] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.535451] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.535454] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.535455] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.535456] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.535462] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.535533] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.535534] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.535536] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.535538] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.535539] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.535540] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.535542] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.535543] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.535544] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.535546] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.535547] PKRU: 55555554 <4> [297.535548] Call Trace: <4> [297.535549] <TASK> <4> [297.535553] ? lock_sync+0x100/0x100 <4> [297.535557] ? lock_release+0xd0/0x2b0 <4> [297.535563] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.535569] process_one_work+0x239/0x760 <4> [297.535574] worker_thread+0x200/0x3f0 <4> [297.535577] ? __pfx_worker_thread+0x10/0x10 <4> [297.535579] kthread+0x10d/0x150 <4> [297.535582] ? __pfx_kthread+0x10/0x10 <4> [297.535586] ret_from_fork+0x3d4/0x480 <4> [297.535588] ? __pfx_kthread+0x10/0x10 <4> [297.535591] ret_from_fork_asm+0x1a/0x30 <4> [297.535599] </TASK> <4> [297.535600] irq event stamp: 974541 <4> [297.535601] hardirqs last enabled at (974547): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.535603] hardirqs last disabled at (974552): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.535606] softirqs last enabled at (971572): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.535608] softirqs last disabled at (971565): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.535610] ---[ end trace 0000000000000000 ]--- <7> [297.535624] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.535730] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [297.535829] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.535900] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.535906] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.535953] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.536247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.536339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.536440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.536528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.536615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.536701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.536788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.536875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.536956] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.537052] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.537172] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.538179] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.548440] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.548689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.549653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.549733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.549810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.549886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.549963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.550039] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.550115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.550193] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.550271] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.550348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.550423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.550512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.550587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.550662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.550737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.550812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.550886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.550959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.551033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.551115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.551200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.551281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.551360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.551443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.551522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.551601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.551679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.551762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.551845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.551926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.552007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.552089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.552176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.552261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.552346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.552428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.552521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.552604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.552686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.552767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.552850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.552933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.553013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.553092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.553174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.553252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.553334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.553415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.553505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.553583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.553660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.553741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.553817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.553895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.553972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.554050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.554128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.554207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.554290] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.554401] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.554405] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40166, lrc_seqno=40166, guc_id=0, flags=0x73 in no process [-1] <7> [297.554407] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.555324] ------------[ cut here ]------------ <4> [297.555326] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.555327] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.555402] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.555472] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.555480] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.555483] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.555484] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.555486] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.555491] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.555561] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.555563] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.555565] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.555566] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.555568] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.555569] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.555570] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.555571] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.555573] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.555574] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.555576] PKRU: 55555554 <4> [297.555577] Call Trace: <4> [297.555578] <TASK> <4> [297.555582] ? lock_sync+0x100/0x100 <4> [297.555586] ? lock_release+0xd0/0x2b0 <4> [297.555592] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.555597] process_one_work+0x239/0x760 <4> [297.555603] worker_thread+0x200/0x3f0 <4> [297.555606] ? __pfx_worker_thread+0x10/0x10 <4> [297.555608] kthread+0x10d/0x150 <4> [297.555611] ? __pfx_kthread+0x10/0x10 <4> [297.555615] ret_from_fork+0x3d4/0x480 <4> [297.555617] ? __pfx_kthread+0x10/0x10 <4> [297.555620] ret_from_fork_asm+0x1a/0x30 <4> [297.555627] </TASK> <4> [297.555629] irq event stamp: 977643 <4> [297.555630] hardirqs last enabled at (977649): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.555633] hardirqs last disabled at (977654): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.555635] softirqs last enabled at (977098): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.555637] softirqs last disabled at (977091): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.555640] ---[ end trace 0000000000000000 ]--- <7> [297.555654] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.555757] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [297.555855] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.555927] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.555934] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.556476] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.556698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.556795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.556890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.556977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.557064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.557149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.557235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.557322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.557404] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.557513] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.557640] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.558652] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.568441] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.568690] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.569850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.569930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.570007] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.570084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.570162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.570240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.570318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.570396] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.570485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.570569] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.570645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.570719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.570792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.570864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.570938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.571011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.571085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.571160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.571233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.571315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.571399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.571496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.571587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.571666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.571743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.571821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.571897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.571976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.572059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.572142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.572223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.572304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.572390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.572488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.572581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.572662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.572742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.572819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.572896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.572972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.573056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.573138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.573219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.573298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.573378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.573465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.573552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.573629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.573711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.573788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.573865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.573944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.574021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.574098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.574174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.574251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.574327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.574402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.574499] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.574623] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.574627] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40166, lrc_seqno=40166, guc_id=0, flags=0x73 in no process [-1] <7> [297.574630] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.574688] ------------[ cut here ]------------ <4> [297.574690] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.574691] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.574762] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.574829] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.574837] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.574840] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.574841] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.574843] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.574848] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.574918] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.574920] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.574922] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.574924] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.574925] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.574926] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.574928] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.574929] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.574931] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.574932] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.574933] PKRU: 55555554 <4> [297.574935] Call Trace: <4> [297.574936] <TASK> <4> [297.574940] ? lock_sync+0x100/0x100 <4> [297.574944] ? lock_release+0xd0/0x2b0 <4> [297.574950] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.574955] process_one_work+0x239/0x760 <4> [297.574962] worker_thread+0x200/0x3f0 <4> [297.574965] ? __pfx_worker_thread+0x10/0x10 <4> [297.574967] kthread+0x10d/0x150 <4> [297.574970] ? __pfx_kthread+0x10/0x10 <4> [297.574974] ret_from_fork+0x3d4/0x480 <4> [297.574976] ? __pfx_kthread+0x10/0x10 <4> [297.574979] ret_from_fork_asm+0x1a/0x30 <4> [297.574987] </TASK> <4> [297.574988] irq event stamp: 1835881 <4> [297.574989] hardirqs last enabled at (1835887): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.574992] hardirqs last disabled at (1835892): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.574994] softirqs last enabled at (1835008): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.574997] softirqs last disabled at (1834993): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.574999] ---[ end trace 0000000000000000 ]--- <5> [297.576255] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40221, lrc_seqno=40221, guc_id=0, flags=0x73 in no process [-1] <7> [297.576258] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.576322] ------------[ cut here ]------------ <4> [297.576323] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.576325] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.576398] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.576484] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.576492] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.576495] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.576496] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.576498] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.576503] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.576573] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.576575] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.576577] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.576578] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.576579] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.576581] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.576582] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.576583] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.576585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.576586] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.576588] PKRU: 55555554 <4> [297.576589] Call Trace: <4> [297.576590] <TASK> <4> [297.576594] ? lock_sync+0x100/0x100 <4> [297.576598] ? lock_release+0xd0/0x2b0 <4> [297.576603] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.576609] process_one_work+0x239/0x760 <4> [297.576615] worker_thread+0x200/0x3f0 <4> [297.576618] ? __pfx_worker_thread+0x10/0x10 <4> [297.576620] kthread+0x10d/0x150 <4> [297.576623] ? __pfx_kthread+0x10/0x10 <4> [297.576627] ret_from_fork+0x3d4/0x480 <4> [297.576628] ? __pfx_kthread+0x10/0x10 <4> [297.576632] ret_from_fork_asm+0x1a/0x30 <4> [297.576639] </TASK> <4> [297.576640] irq event stamp: 1837777 <4> [297.576641] hardirqs last enabled at (1837783): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.576644] hardirqs last disabled at (1837788): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.576646] softirqs last enabled at (1835008): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.576649] softirqs last disabled at (1834993): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.576651] ---[ end trace 0000000000000000 ]--- <7> [297.576666] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.576772] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [297.576872] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.576946] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.576953] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.577002] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.577318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.577411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.577520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.577613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.577704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.577792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.577880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.577967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.578049] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.578144] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.578262] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.579273] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.589439] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.589688] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.590709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.590789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.590867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.590945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.591021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.591098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.591173] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.591249] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.591324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.591402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.591494] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.591572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.591647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.591722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.591796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.591871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.591946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.592025] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.592102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.592183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.592265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.592345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.592427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.592525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.592605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.592687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.592766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.592848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.592928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.593007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.593086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.593165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.593250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.593334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.593422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.593520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.593601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.593681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.593760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.593837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.593917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.593996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.594078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.594157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.594242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.594321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.594399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.594487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.594566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.594647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.594728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.594810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.594889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.594969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.595046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.595126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.595203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.595279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.595360] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.595479] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.595483] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40221, lrc_seqno=40221, guc_id=0, flags=0x73 in no process [-1] <7> [297.595485] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.595545] ------------[ cut here ]------------ <4> [297.595546] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.595547] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:1/120 <4> [297.595620] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.595685] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.595693] CPU: 4 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.595696] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.595697] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.595698] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.595703] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.595774] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.595776] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.595778] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.595779] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.595781] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.595782] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.595783] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.595785] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.595786] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.595788] CR2: 000072fba006c418 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.595789] PKRU: 55555554 <4> [297.595790] Call Trace: <4> [297.595791] <TASK> <4> [297.595795] ? lock_sync+0x100/0x100 <4> [297.595800] ? lock_release+0xd0/0x2b0 <4> [297.595805] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.595811] process_one_work+0x239/0x760 <4> [297.595817] worker_thread+0x200/0x3f0 <4> [297.595820] ? __pfx_worker_thread+0x10/0x10 <4> [297.595822] kthread+0x10d/0x150 <4> [297.595825] ? __pfx_kthread+0x10/0x10 <4> [297.595829] ret_from_fork+0x3d4/0x480 <4> [297.595831] ? __pfx_kthread+0x10/0x10 <4> [297.595834] ret_from_fork_asm+0x1a/0x30 <4> [297.595842] </TASK> <4> [297.595843] irq event stamp: 1020423 <4> [297.595844] hardirqs last enabled at (1020429): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.595847] hardirqs last disabled at (1020434): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.595849] softirqs last enabled at (1019576): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.595852] softirqs last disabled at (1019553): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.595854] ---[ end trace 0000000000000000 ]--- <6> [297.595856] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.595925] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.595931] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.597163] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.597475] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.597584] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.597787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.597879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.597972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.598061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.598146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.598231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.598316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.598403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.598494] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.598591] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.598709] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.599715] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.610438] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.610683] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.611776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.611858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.611933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.612005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.612076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.612146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.612215] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.612285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.612354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.612422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.612507] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.612579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.612654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.612727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.612800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.612873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.612947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.613019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.613093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.613175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.613260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.613343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.613424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.613518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.613597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.613676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.613753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.613835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.613921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.614004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.614087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.614167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.614252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.614337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.614422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.614516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.614596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.614672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.614748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.614824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.614903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.614981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.615059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.615135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.615215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.615293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.615369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.615448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.615530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.615611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.615689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.615770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.615847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.615925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.616002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.616081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.616162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.616240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.616322] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.617274] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.617278] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40221, lrc_seqno=40221, guc_id=0, flags=0x73 in no process [-1] <7> [297.617281] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.617341] ------------[ cut here ]------------ <4> [297.617343] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.617344] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.617415] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.617485] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.617493] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.617496] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.617497] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.617499] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.617504] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.617574] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.617575] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.617577] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.617579] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.617580] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.617581] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.617582] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.617584] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.617585] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.617587] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.617588] PKRU: 55555554 <4> [297.617589] Call Trace: <4> [297.617590] <TASK> <4> [297.617594] ? lock_sync+0x100/0x100 <4> [297.617599] ? lock_release+0xd0/0x2b0 <4> [297.617604] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.617610] process_one_work+0x239/0x760 <4> [297.617616] worker_thread+0x200/0x3f0 <4> [297.617619] ? __pfx_worker_thread+0x10/0x10 <4> [297.617621] kthread+0x10d/0x150 <4> [297.617624] ? __pfx_kthread+0x10/0x10 <4> [297.617628] ret_from_fork+0x3d4/0x480 <4> [297.617630] ? __pfx_kthread+0x10/0x10 <4> [297.617633] ret_from_fork_asm+0x1a/0x30 <4> [297.617641] </TASK> <4> [297.617642] irq event stamp: 981059 <4> [297.617643] hardirqs last enabled at (981065): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.617646] hardirqs last disabled at (981070): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.617648] softirqs last enabled at (980402): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.617650] softirqs last disabled at (980397): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.617652] ---[ end trace 0000000000000000 ]--- <7> [297.617666] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.617767] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [297.618216] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40276, lrc_seqno=40276, guc_id=0, flags=0x73 in no process [-1] <7> [297.618218] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.618279] ------------[ cut here ]------------ <4> [297.618280] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.618281] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.618354] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.618418] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.618425] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.618428] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.618429] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.618434] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.618440] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.618510] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.618511] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.618513] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.618515] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.618516] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.618517] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.618518] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.618520] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.618521] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.618522] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.618524] PKRU: 55555554 <4> [297.618525] Call Trace: <4> [297.618526] <TASK> <4> [297.618530] ? lock_sync+0x100/0x100 <4> [297.618534] ? lock_release+0xd0/0x2b0 <4> [297.618539] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.618545] process_one_work+0x239/0x760 <4> [297.618551] worker_thread+0x200/0x3f0 <4> [297.618553] ? __pfx_worker_thread+0x10/0x10 <4> [297.618556] kthread+0x10d/0x150 <4> [297.618558] ? __pfx_kthread+0x10/0x10 <4> [297.618562] ret_from_fork+0x3d4/0x480 <4> [297.618564] ? __pfx_kthread+0x10/0x10 <4> [297.618567] ret_from_fork_asm+0x1a/0x30 <4> [297.618575] </TASK> <4> [297.618576] irq event stamp: 982965 <4> [297.618577] hardirqs last enabled at (982971): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.618580] hardirqs last disabled at (982976): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.618582] softirqs last enabled at (982786): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.618584] softirqs last disabled at (982779): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.618586] ---[ end trace 0000000000000000 ]--- <6> [297.618588] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.618658] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.618663] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.618879] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.619084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.619175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.619268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.619357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.619448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.619533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.619620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.619708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.619790] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.619885] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.620003] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.621011] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.631438] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.631688] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.632695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.632776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.632853] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.632930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.633006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.633080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.633159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.633239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.633316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.633392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.633477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.633552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.633625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.633700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.633775] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.633849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.633923] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.633998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.634073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.634155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.634241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.634323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.634405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.634494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.634571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.634649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.634724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.634806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.634889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.634969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.635049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.635129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.635213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.635298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.635382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.635472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.635552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.635630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.635708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.635789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.635872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.635952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.636031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.636108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.636189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.636266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.636343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.636419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.636509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.636590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.636670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.636751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.636828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.636906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.636982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.637060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.637135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.637213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.637297] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.637408] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.637412] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40276, lrc_seqno=40276, guc_id=0, flags=0x73 in no process [-1] <7> [297.637415] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.637495] ------------[ cut here ]------------ <4> [297.637496] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.637498] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:1/120 <4> [297.637570] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.637634] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.637642] CPU: 4 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.637645] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.637646] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.637647] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.637652] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.637722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.637723] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.637726] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.637727] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.637728] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.637729] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.637731] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.637732] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.637734] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.637735] CR2: 000072fba006c418 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [297.637736] PKRU: 55555554 <4> [297.637738] Call Trace: <4> [297.637739] <TASK> <4> [297.637743] ? lock_sync+0x100/0x100 <4> [297.637747] ? lock_release+0xd0/0x2b0 <4> [297.637752] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.637758] process_one_work+0x239/0x760 <4> [297.637764] worker_thread+0x200/0x3f0 <4> [297.637767] ? __pfx_worker_thread+0x10/0x10 <4> [297.637769] kthread+0x10d/0x150 <4> [297.637772] ? __pfx_kthread+0x10/0x10 <4> [297.637776] ret_from_fork+0x3d4/0x480 <4> [297.637778] ? __pfx_kthread+0x10/0x10 <4> [297.637782] ret_from_fork_asm+0x1a/0x30 <4> [297.637789] </TASK> <4> [297.637790] irq event stamp: 1023823 <4> [297.637791] hardirqs last enabled at (1023829): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.637794] hardirqs last disabled at (1023834): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.637796] softirqs last enabled at (1023032): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.637799] softirqs last disabled at (1023019): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.637801] ---[ end trace 0000000000000000 ]--- <6> [297.637803] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.637872] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.637878] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.639128] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.639232] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.639339] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.639560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.639653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.639751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.639842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.639930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.640016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.640102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.640189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.640270] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.640365] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.640491] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.641528] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.652250] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.652503] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.653636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.653718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.653793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.653869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.653945] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.654021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.654101] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.654180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.654257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.654333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.654405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.654491] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.654575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.654650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.654726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.654800] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.654873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.654947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.655021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.655103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.655185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.655266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.655346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.655424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.655521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.655608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.655689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.655771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.655853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.655931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.656010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.656089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.656174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.656258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.656341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.656425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.656530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.656621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.656699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.656778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.656858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.656936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.657014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.657091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.657172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.657248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.657325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.657404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.657500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.657589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.657667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.657747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.657825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.657903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.657978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.658055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.658130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.658209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.658293] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.658407] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.658411] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40276, lrc_seqno=40276, guc_id=0, flags=0x73 in no process [-1] <7> [297.658414] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.659347] ------------[ cut here ]------------ <4> [297.659348] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.659350] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.659423] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.659507] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.659516] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.659519] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.659520] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.659522] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.659528] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.659598] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.659600] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.659602] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.659603] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.659605] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.659606] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.659607] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.659608] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.659610] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.659611] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.659613] PKRU: 55555554 <4> [297.659614] Call Trace: <4> [297.659615] <TASK> <4> [297.659619] ? lock_sync+0x100/0x100 <4> [297.659624] ? lock_release+0xd0/0x2b0 <4> [297.659629] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.659635] process_one_work+0x239/0x760 <4> [297.659642] worker_thread+0x200/0x3f0 <4> [297.659645] ? __pfx_worker_thread+0x10/0x10 <4> [297.659647] kthread+0x10d/0x150 <4> [297.659650] ? __pfx_kthread+0x10/0x10 <4> [297.659654] ret_from_fork+0x3d4/0x480 <4> [297.659656] ? __pfx_kthread+0x10/0x10 <4> [297.659659] ret_from_fork_asm+0x1a/0x30 <4> [297.659667] </TASK> <4> [297.659668] irq event stamp: 1842827 <4> [297.659669] hardirqs last enabled at (1842833): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.659672] hardirqs last disabled at (1842838): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.659674] softirqs last enabled at (1842030): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.659677] softirqs last disabled at (1842023): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.659679] ---[ end trace 0000000000000000 ]--- <7> [297.660081] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.660185] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [297.660290] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40331, lrc_seqno=40331, guc_id=0, flags=0x73 in no process [-1] <7> [297.660292] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.660352] ------------[ cut here ]------------ <4> [297.660353] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.660354] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.660428] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.660511] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.660519] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.660522] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.660523] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.660524] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.660529] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.660600] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.660602] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.660604] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.660605] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.660607] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.660608] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.660609] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.660610] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.660612] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.660613] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.660615] PKRU: 55555554 <4> [297.660616] Call Trace: <4> [297.660617] <TASK> <4> [297.660621] ? lock_sync+0x100/0x100 <4> [297.660625] ? lock_release+0xd0/0x2b0 <4> [297.660630] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.660636] process_one_work+0x239/0x760 <4> [297.660642] worker_thread+0x200/0x3f0 <4> [297.660645] ? __pfx_worker_thread+0x10/0x10 <4> [297.660647] kthread+0x10d/0x150 <4> [297.660650] ? __pfx_kthread+0x10/0x10 <4> [297.660654] ret_from_fork+0x3d4/0x480 <4> [297.660656] ? __pfx_kthread+0x10/0x10 <4> [297.660659] ret_from_fork_asm+0x1a/0x30 <4> [297.660666] </TASK> <4> [297.660667] irq event stamp: 1844759 <4> [297.660669] hardirqs last enabled at (1844765): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.660671] hardirqs last disabled at (1844770): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.660673] softirqs last enabled at (1842030): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.660676] softirqs last disabled at (1842023): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.660678] ---[ end trace 0000000000000000 ]--- <6> [297.660680] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.660749] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.660755] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.660802] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.661006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.661096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.661189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.661277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.661363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.661457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.661565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.661652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.661738] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.661837] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.661956] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.662969] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.673439] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.673689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.674717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.674810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.674895] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.674973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.675050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.675125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.675201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.675276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.675352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.675425] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.675518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.675601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.675674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.675748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.675823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.675897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.675971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.676048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.676125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.676208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.676289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.676369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.676459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.676546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.676624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.676702] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.676782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.676866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.676949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.677029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.677109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.677189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.677272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.677355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.677445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.677536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.677618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.677698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.677776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.677853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.677938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.678018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.678098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.678177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.678259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.678336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.678414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.678505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.678595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.678672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.678747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.678826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.678904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.678986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.679065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.679144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.679221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.679299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.679379] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.679507] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.679511] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40331, lrc_seqno=40331, guc_id=0, flags=0x73 in no process [-1] <7> [297.679514] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.679573] ------------[ cut here ]------------ <4> [297.679574] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.679575] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.679647] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.679710] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.679718] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.679721] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.679722] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.679724] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.679729] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.679798] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.679800] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.679802] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.679803] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.679805] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.679806] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.679807] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.679808] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.679810] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.679811] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.679813] PKRU: 55555554 <4> [297.679814] Call Trace: <4> [297.679815] <TASK> <4> [297.679819] ? lock_sync+0x100/0x100 <4> [297.679823] ? lock_release+0xd0/0x2b0 <4> [297.679828] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.679834] process_one_work+0x239/0x760 <4> [297.679840] worker_thread+0x200/0x3f0 <4> [297.679843] ? __pfx_worker_thread+0x10/0x10 <4> [297.679845] kthread+0x10d/0x150 <4> [297.679848] ? __pfx_kthread+0x10/0x10 <4> [297.679852] ret_from_fork+0x3d4/0x480 <4> [297.679854] ? __pfx_kthread+0x10/0x10 <4> [297.679857] ret_from_fork_asm+0x1a/0x30 <4> [297.679864] </TASK> <4> [297.679865] irq event stamp: 1847831 <4> [297.679867] hardirqs last enabled at (1847837): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.679869] hardirqs last disabled at (1847842): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.679872] softirqs last enabled at (1846966): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.679874] softirqs last disabled at (1846961): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.679876] ---[ end trace 0000000000000000 ]--- <6> [297.679878] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.679950] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.679956] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.680362] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.680577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.680666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.680758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.680846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.680934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.681020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.681106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.681193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.681275] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.681370] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.681502] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.682524] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.693247] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.693488] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.694551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.694623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.694693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.694765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.694836] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.694906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.694978] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.695049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.695119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.695189] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.695258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.695325] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.695393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.695476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.695561] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.695635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.695709] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.695783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.695857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.695939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.696021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.696104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.696186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.696265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.696342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.696419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.696513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.696608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.696692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.696773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.696851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.696931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.697016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.697101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.697184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.697265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.697344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.697421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.697516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.697602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.697682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.697765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.697847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.697925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.698008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.698087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.698164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.698241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.698319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.698396] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.698484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.698580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.698663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.698745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.698823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.698902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.698980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.699056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.699136] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.699248] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.699252] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40331, lrc_seqno=40331, guc_id=0, flags=0x73 in no process [-1] <7> [297.699255] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.699313] ------------[ cut here ]------------ <4> [297.699315] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.699316] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.699388] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.699487] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.699495] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.699498] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.699499] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.699501] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.699506] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.699576] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.699578] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.699580] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.699582] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.699583] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.699584] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.699585] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.699587] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.699588] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.699590] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.699591] PKRU: 55555554 <4> [297.699592] Call Trace: <4> [297.699593] <TASK> <4> [297.699597] ? lock_sync+0x100/0x100 <4> [297.699601] ? lock_release+0xd0/0x2b0 <4> [297.699607] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.699613] process_one_work+0x239/0x760 <4> [297.699619] worker_thread+0x200/0x3f0 <4> [297.699622] ? __pfx_worker_thread+0x10/0x10 <4> [297.699624] kthread+0x10d/0x150 <4> [297.699627] ? __pfx_kthread+0x10/0x10 <4> [297.699631] ret_from_fork+0x3d4/0x480 <4> [297.699633] ? __pfx_kthread+0x10/0x10 <4> [297.699636] ret_from_fork_asm+0x1a/0x30 <4> [297.699643] </TASK> <4> [297.699644] irq event stamp: 1850917 <4> [297.699646] hardirqs last enabled at (1850923): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.699648] hardirqs last disabled at (1850928): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.699651] softirqs last enabled at (1849990): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.699653] softirqs last disabled at (1849983): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.699655] ---[ end trace 0000000000000000 ]--- <7> [297.700942] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.701050] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [297.701150] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40386, lrc_seqno=40386, guc_id=0, flags=0x73 in no process [-1] <7> [297.701152] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.701211] ------------[ cut here ]------------ <4> [297.701212] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.701213] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:3/193 <4> [297.701285] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.701347] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.701355] CPU: 6 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.701357] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.701358] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.701360] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.701364] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.701440] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.701442] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.701446] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.701449] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.701451] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.701453] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.701456] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.701458] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [297.701461] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.701463] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [297.701466] PKRU: 55555554 <4> [297.701468] Call Trace: <4> [297.701470] <TASK> <4> [297.701477] ? lock_sync+0x100/0x100 <4> [297.701485] ? lock_release+0xd0/0x2b0 <4> [297.701495] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.701505] process_one_work+0x239/0x760 <4> [297.701515] worker_thread+0x200/0x3f0 <4> [297.701520] ? __pfx_worker_thread+0x10/0x10 <4> [297.701524] kthread+0x10d/0x150 <4> [297.701529] ? __pfx_kthread+0x10/0x10 <4> [297.701535] ret_from_fork+0x3d4/0x480 <4> [297.701538] ? __pfx_kthread+0x10/0x10 <4> [297.701544] ret_from_fork_asm+0x1a/0x30 <4> [297.701557] </TASK> <4> [297.701559] irq event stamp: 1852827 <4> [297.701561] hardirqs last enabled at (1852833): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.701565] hardirqs last disabled at (1852838): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.701569] softirqs last enabled at (1849990): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.701573] softirqs last disabled at (1849983): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.701576] ---[ end trace 0000000000000000 ]--- <6> [297.701579] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.701658] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.701664] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.701792] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.701999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.702090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.702185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.702273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.702364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.702461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.702551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.702640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.702723] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.702818] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.702936] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.703945] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.714669] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.714919] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.715883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.715964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.716042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.716120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.716197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.716274] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.716350] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.716426] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.716525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.716653] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.716755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.716833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.716908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.716983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.717057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.717131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.717205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.717278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.717350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.717439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.717522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.717601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.717681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.717759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.717837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.717915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.717991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.718071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.718155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.718235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.718323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.718403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.718498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.718583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.718667] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.718748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.718833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.718914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.718992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.719069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.719148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.719227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.719305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.719380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.719469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.719547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.719623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.719699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.719778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.719854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.719934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.720016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.720096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.720175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.720252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.720331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.720407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.720494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.720575] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.720687] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.720691] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40386, lrc_seqno=40386, guc_id=0, flags=0x73 in no process [-1] <7> [297.720693] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.720751] ------------[ cut here ]------------ <4> [297.720752] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.720754] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.720827] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.720890] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.720899] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.720901] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.720902] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.720904] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.720909] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.720980] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.720981] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.720984] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.720985] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.720986] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.720988] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.720989] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.720990] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.720992] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.720993] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.720995] PKRU: 55555554 <4> [297.720996] Call Trace: <4> [297.720997] <TASK> <4> [297.721001] ? lock_sync+0x100/0x100 <4> [297.721005] ? lock_release+0xd0/0x2b0 <4> [297.721011] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.721016] process_one_work+0x239/0x760 <4> [297.721023] worker_thread+0x200/0x3f0 <4> [297.721025] ? __pfx_worker_thread+0x10/0x10 <4> [297.721028] kthread+0x10d/0x150 <4> [297.721031] ? __pfx_kthread+0x10/0x10 <4> [297.721034] ret_from_fork+0x3d4/0x480 <4> [297.721036] ? __pfx_kthread+0x10/0x10 <4> [297.721040] ret_from_fork_asm+0x1a/0x30 <4> [297.721047] </TASK> <4> [297.721048] irq event stamp: 987109 <4> [297.721049] hardirqs last enabled at (987115): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.721052] hardirqs last disabled at (987120): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.721054] softirqs last enabled at (986230): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.721057] softirqs last disabled at (986225): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.721059] ---[ end trace 0000000000000000 ]--- <6> [297.721061] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.721131] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.721137] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.722378] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.722493] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.722610] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.722815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.722908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.723002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.723089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.723175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.723259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.723344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.723436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.723518] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.723612] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.723728] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.724734] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.735449] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.735697] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.736880] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.736958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.737028] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.737099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.737168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.737237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.737305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.737374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.737446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.737525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.737602] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.737678] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.737753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.737828] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.737903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.737977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.738051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.738124] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.738198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.738284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.738367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.738449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.738530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.738606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.738682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.738757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.738835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.738918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.739002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.739082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.739163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.739244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.739328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.739412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.739511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.739596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.739679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.739758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.739837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.739914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.739995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.740073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.740151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.740228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.740308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.740386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.740472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.740550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.740631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.740710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.740788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.740869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.740946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.741024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.741100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.741178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.741254] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.741332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.741413] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.741540] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.741544] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40386, lrc_seqno=40386, guc_id=0, flags=0x73 in no process [-1] <7> [297.741546] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.741604] ------------[ cut here ]------------ <4> [297.741606] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.741607] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:1/120 <4> [297.741677] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.741740] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.741748] CPU: 4 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.741751] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.741752] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.741754] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.741759] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.741828] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.741830] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.741832] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.741833] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.741835] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.741836] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.741837] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.741838] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.741840] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.741841] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.741843] PKRU: 55555554 <4> [297.741844] Call Trace: <4> [297.741845] <TASK> <4> [297.741849] ? lock_sync+0x100/0x100 <4> [297.741853] ? lock_release+0xd0/0x2b0 <4> [297.741859] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.741864] process_one_work+0x239/0x760 <4> [297.741870] worker_thread+0x200/0x3f0 <4> [297.741873] ? __pfx_worker_thread+0x10/0x10 <4> [297.741876] kthread+0x10d/0x150 <4> [297.741878] ? __pfx_kthread+0x10/0x10 <4> [297.741882] ret_from_fork+0x3d4/0x480 <4> [297.741884] ? __pfx_kthread+0x10/0x10 <4> [297.741888] ret_from_fork_asm+0x1a/0x30 <4> [297.741895] </TASK> <4> [297.741896] irq event stamp: 1030933 <4> [297.741897] hardirqs last enabled at (1030939): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.741900] hardirqs last disabled at (1030944): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.741902] softirqs last enabled at (1029982): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.741905] softirqs last disabled at (1029975): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.741907] ---[ end trace 0000000000000000 ]--- <5> [297.743262] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40441, lrc_seqno=40441, guc_id=0, flags=0x73 in no process [-1] <7> [297.743264] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.743325] ------------[ cut here ]------------ <4> [297.743326] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.743327] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:1/120 <4> [297.743397] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.743464] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.743472] CPU: 4 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.743475] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.743476] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.743477] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.743482] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.743550] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.743552] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.743554] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.743555] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.743556] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.743558] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.743559] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.743560] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.743561] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.743563] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.743564] PKRU: 55555554 <4> [297.743565] Call Trace: <4> [297.743566] <TASK> <4> [297.743570] ? lock_sync+0x100/0x100 <4> [297.743574] ? lock_release+0xd0/0x2b0 <4> [297.743579] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.743585] process_one_work+0x239/0x760 <4> [297.743591] worker_thread+0x200/0x3f0 <4> [297.743594] ? __pfx_worker_thread+0x10/0x10 <4> [297.743596] kthread+0x10d/0x150 <4> [297.743599] ? __pfx_kthread+0x10/0x10 <4> [297.743603] ret_from_fork+0x3d4/0x480 <4> [297.743605] ? __pfx_kthread+0x10/0x10 <4> [297.743608] ret_from_fork_asm+0x1a/0x30 <4> [297.743615] </TASK> <4> [297.743616] irq event stamp: 1032847 <4> [297.743617] hardirqs last enabled at (1032853): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.743620] hardirqs last disabled at (1032858): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.743622] softirqs last enabled at (1029982): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.743625] softirqs last disabled at (1029975): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.743627] ---[ end trace 0000000000000000 ]--- <6> [297.743628] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.743698] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.743705] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.743860] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.743963] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.744071] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.744278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.744371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.744485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.744575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.744661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.744747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.744835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.744923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.745007] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.745105] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.745224] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.746236] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.756437] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.756686] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.757697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.757776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.757852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.757931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.758010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.758087] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.758164] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.758239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.758314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.758389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.758475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.758549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.758623] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.758697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.758771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.758844] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.758917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.758990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.759063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.759143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.759223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.759302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.759382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.759477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.759559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.759639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.759719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.759799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.759879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.759959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.760038] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.760116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.760204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.760290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.760379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.760472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.760556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.760636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.760717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.760793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.760874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.760953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.761031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.761107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.761187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.761265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.761343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.761420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.761515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.761597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.761677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.761759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.761838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.761917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.761994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.762077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.762158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.762247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.762329] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.762449] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.762453] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40441, lrc_seqno=40441, guc_id=0, flags=0x73 in no process [-1] <7> [297.762456] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.762516] ------------[ cut here ]------------ <4> [297.762518] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.762519] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.762591] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.762655] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.762664] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.762667] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.762668] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.762669] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.762674] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.762745] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.762747] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.762749] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.762751] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.762752] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.762753] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.762755] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.762756] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.762758] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.762759] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.762761] PKRU: 55555554 <4> [297.762762] Call Trace: <4> [297.762763] <TASK> <4> [297.762767] ? lock_sync+0x100/0x100 <4> [297.762771] ? lock_release+0xd0/0x2b0 <4> [297.762777] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.762783] process_one_work+0x239/0x760 <4> [297.762789] worker_thread+0x200/0x3f0 <4> [297.762792] ? __pfx_worker_thread+0x10/0x10 <4> [297.762794] kthread+0x10d/0x150 <4> [297.762797] ? __pfx_kthread+0x10/0x10 <4> [297.762801] ret_from_fork+0x3d4/0x480 <4> [297.762803] ? __pfx_kthread+0x10/0x10 <4> [297.762806] ret_from_fork_asm+0x1a/0x30 <4> [297.762814] </TASK> <4> [297.762815] irq event stamp: 991573 <4> [297.762816] hardirqs last enabled at (991579): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.762819] hardirqs last disabled at (991584): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.762821] softirqs last enabled at (990540): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.762823] softirqs last disabled at (990533): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.762826] ---[ end trace 0000000000000000 ]--- <6> [297.762828] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.762898] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.762903] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.763309] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.763522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.763606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.763696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.763784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.763873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.763960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.764047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.764136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.764219] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.764315] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.764442] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.765470] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.776193] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.776433] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.777503] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.777575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.777646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.777718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.777789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.777861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.777932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.778001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.778070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.778139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.778208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.778277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.778344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.778412] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.778498] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.778572] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.778646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.778721] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.778795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.778877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.778958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.779040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.779122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.779201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.779279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.779364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.779451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.779531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.779611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.779690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.779769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.779848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.779933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.780021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.780107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.780191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.780272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.780350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.780437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.780518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.780599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.780679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.780757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.780834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.780916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.780995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.781076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.781155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.781235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.781313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.781390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.781480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.781559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.781639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.781719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.781799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.781876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.781952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.782032] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.782145] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.782149] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40441, lrc_seqno=40441, guc_id=0, flags=0x73 in no process [-1] <7> [297.782152] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.782210] ------------[ cut here ]------------ <4> [297.782211] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.782212] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#3: kworker/u64:14/2466 <4> [297.782283] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.782348] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.782356] CPU: 3 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.782359] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.782360] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.782361] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.782367] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.782439] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.782440] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.782443] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.782444] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.782446] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.782447] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.782448] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.782450] FS: 0000000000000000(0000) GS:ffff8888dae17000(0000) knlGS:0000000000000000 <4> [297.782452] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.782453] CR2: 000077f1ad1e62f4 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.782455] PKRU: 55555554 <4> [297.782456] Call Trace: <4> [297.782457] <TASK> <4> [297.782461] ? lock_sync+0x100/0x100 <4> [297.782465] ? lock_release+0xd0/0x2b0 <4> [297.782471] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.782477] process_one_work+0x239/0x760 <4> [297.782483] worker_thread+0x200/0x3f0 <4> [297.782486] ? __pfx_worker_thread+0x10/0x10 <4> [297.782488] kthread+0x10d/0x150 <4> [297.782491] ? __pfx_kthread+0x10/0x10 <4> [297.782495] ret_from_fork+0x3d4/0x480 <4> [297.782497] ? __pfx_kthread+0x10/0x10 <4> [297.782500] ret_from_fork_asm+0x1a/0x30 <4> [297.782508] </TASK> <4> [297.782509] irq event stamp: 994671 <4> [297.782510] hardirqs last enabled at (994677): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.782513] hardirqs last disabled at (994682): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.782515] softirqs last enabled at (993562): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.782518] softirqs last disabled at (993553): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.782520] ---[ end trace 0000000000000000 ]--- <7> [297.783545] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.783647] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [297.784064] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40496, lrc_seqno=40496, guc_id=0, flags=0x73 in no process [-1] <7> [297.784068] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.784128] ------------[ cut here ]------------ <4> [297.784129] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.784131] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.784205] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.784269] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.784277] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.784280] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.784282] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.784283] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.784288] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.784359] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.784361] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.784363] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.784365] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.784366] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.784367] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.784368] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.784369] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.784371] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.784372] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.784374] PKRU: 55555554 <4> [297.784375] Call Trace: <4> [297.784376] <TASK> <4> [297.784380] ? lock_sync+0x100/0x100 <4> [297.784384] ? lock_release+0xd0/0x2b0 <4> [297.784390] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.784395] process_one_work+0x239/0x760 <4> [297.784401] worker_thread+0x200/0x3f0 <4> [297.784404] ? __pfx_worker_thread+0x10/0x10 <4> [297.784407] kthread+0x10d/0x150 <4> [297.784409] ? __pfx_kthread+0x10/0x10 <4> [297.784413] ret_from_fork+0x3d4/0x480 <4> [297.784415] ? __pfx_kthread+0x10/0x10 <4> [297.784419] ret_from_fork_asm+0x1a/0x30 <4> [297.784433] </TASK> <4> [297.784434] irq event stamp: 996621 <4> [297.784435] hardirqs last enabled at (996627): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.784438] hardirqs last disabled at (996632): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.784441] softirqs last enabled at (996614): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.784443] softirqs last disabled at (996605): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.784445] ---[ end trace 0000000000000000 ]--- <6> [297.784447] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.784518] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.784523] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.784675] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.784877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.784967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.785060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.785148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.785234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.785319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.785403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.785499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.785582] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.785677] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.785794] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.786805] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.797527] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.797777] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.798736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.798817] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.798893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.798971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.799047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.799123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.799198] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.799273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.799348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.799422] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.799512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.799591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.799668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.799743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.799819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.799894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.799968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.800043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.800116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.800198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.800282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.800364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.800447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.800525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.800603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.800679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.800759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.800842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.800924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.801004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.801084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.801163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.801248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.801333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.801417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.801511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.801591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.801669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.801746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.801824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.801908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.801988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.802067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.802144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.802224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.802302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.802385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.802474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.802554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.802633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.802710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.802790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.802868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.802948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.803029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.803110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.803187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.803264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.803344] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.803461] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.803465] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40496, lrc_seqno=40496, guc_id=0, flags=0x73 in no process [-1] <7> [297.803467] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.803525] ------------[ cut here ]------------ <4> [297.803526] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.803528] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.803600] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.803664] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.803672] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.803675] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.803676] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.803677] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.803682] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.803753] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.803755] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.803757] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.803758] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.803760] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.803761] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.803762] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.803763] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.803765] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.803766] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.803768] PKRU: 55555554 <4> [297.803769] Call Trace: <4> [297.803770] <TASK> <4> [297.803774] ? lock_sync+0x100/0x100 <4> [297.803778] ? lock_release+0xd0/0x2b0 <4> [297.803784] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.803789] process_one_work+0x239/0x760 <4> [297.803795] worker_thread+0x200/0x3f0 <4> [297.803798] ? __pfx_worker_thread+0x10/0x10 <4> [297.803801] kthread+0x10d/0x150 <4> [297.803803] ? __pfx_kthread+0x10/0x10 <4> [297.803807] ret_from_fork+0x3d4/0x480 <4> [297.803809] ? __pfx_kthread+0x10/0x10 <4> [297.803812] ret_from_fork_asm+0x1a/0x30 <4> [297.803820] </TASK> <4> [297.803821] irq event stamp: 999737 <4> [297.803822] hardirqs last enabled at (999743): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.803825] hardirqs last disabled at (999748): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.803827] softirqs last enabled at (998898): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.803829] softirqs last disabled at (998887): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.803832] ---[ end trace 0000000000000000 ]--- <6> [297.803833] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.803903] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.803909] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.804318] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.804524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.804612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.804704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.804792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.804879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.804965] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.805054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.805143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.805226] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.805321] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.805441] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.806446] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.817166] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.817408] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.818472] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.818549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.818620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.818690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.818761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.818830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.818898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.818967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.819036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.819104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.819176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.819247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.819316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.819385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.819465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.819541] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.819614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.819688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.819762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.819844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.819926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.820006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.820087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.820170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.820250] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.820328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.820406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.820495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.820575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.820653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.820731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.820810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.820894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.820976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.821060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.821139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.821219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.821296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.821373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.821453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.821533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.821614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.821695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.821775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.821856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.821934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.822012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.822088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.822166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.822241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.822320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.822403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.822489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.822567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.822644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.822722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.822797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.822872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.822951] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.823060] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.823064] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40496, lrc_seqno=40496, guc_id=0, flags=0x73 in no process [-1] <7> [297.823066] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.823124] ------------[ cut here ]------------ <4> [297.823125] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.823126] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.823197] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.823261] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.823269] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.823272] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.823273] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.823274] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.823279] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.823349] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.823351] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.823353] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.823354] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.823355] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.823357] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.823358] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.823359] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.823361] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.823362] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.823363] PKRU: 55555554 <4> [297.823365] Call Trace: <4> [297.823366] <TASK> <4> [297.823369] ? lock_sync+0x100/0x100 <4> [297.823374] ? lock_release+0xd0/0x2b0 <4> [297.823379] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.823385] process_one_work+0x239/0x760 <4> [297.823391] worker_thread+0x200/0x3f0 <4> [297.823394] ? __pfx_worker_thread+0x10/0x10 <4> [297.823396] kthread+0x10d/0x150 <4> [297.823399] ? __pfx_kthread+0x10/0x10 <4> [297.823402] ret_from_fork+0x3d4/0x480 <4> [297.823404] ? __pfx_kthread+0x10/0x10 <4> [297.823408] ret_from_fork_asm+0x1a/0x30 <4> [297.823415] </TASK> <4> [297.823416] irq event stamp: 1002839 <4> [297.823417] hardirqs last enabled at (1002845): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.823420] hardirqs last disabled at (1002850): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.823422] softirqs last enabled at (1001838): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.823428] softirqs last disabled at (1001831): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.823430] ---[ end trace 0000000000000000 ]--- <5> [297.824715] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40551, lrc_seqno=40551, guc_id=0, flags=0x73 in no process [-1] <7> [297.824724] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.824722] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.824832] ------------[ cut here ]------------ <4> [297.824834] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <7> [297.824828] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [297.824836] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:3/193 <4> [297.824955] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.825046] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.825058] CPU: 12 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.825062] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.825064] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.825066] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.825074] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.825189] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.825192] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.825195] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.825197] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.825199] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.825201] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.825203] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.825205] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [297.825207] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.825209] CR2: 000072fba006e408 CR3: 000000010fbf1004 CR4: 0000000000f72ef0 <4> [297.825211] PKRU: 55555554 <4> [297.825213] Call Trace: <4> [297.825215] <TASK> <4> [297.825221] ? lock_sync+0x100/0x100 <4> [297.825228] ? lock_release+0xd0/0x2b0 <4> [297.825237] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.825246] process_one_work+0x239/0x760 <4> [297.825256] worker_thread+0x200/0x3f0 <4> [297.825261] ? __pfx_worker_thread+0x10/0x10 <4> [297.825264] kthread+0x10d/0x150 <4> [297.825268] ? __pfx_kthread+0x10/0x10 <4> [297.825273] ret_from_fork+0x3d4/0x480 <4> [297.825276] ? __pfx_kthread+0x10/0x10 <4> [297.825280] ret_from_fork_asm+0x1a/0x30 <4> [297.825293] </TASK> <4> [297.825294] irq event stamp: 1855943 <4> [297.825296] hardirqs last enabled at (1855949): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.825300] hardirqs last disabled at (1855954): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.825303] softirqs last enabled at (1854396): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70 <4> [297.825306] softirqs last disabled at (1854394): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120 <4> [297.825309] ---[ end trace 0000000000000000 ]--- <6> [297.825312] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.825434] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.825443] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.825591] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.825794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.825882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.825975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.826062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.826146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.826231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.826315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.826400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.826492] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.826586] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.826707] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.827713] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.838432] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.838682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.839714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.839793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.839869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.839948] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.840027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.840105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.840180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.840254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.840329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.840402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.840485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.840558] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.840631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.840704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.840778] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.840852] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.840924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.840999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.841076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.841161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.841243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.841323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.841403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.841489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.841568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.841649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.841729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.841810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.841891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.841973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.842052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.842135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.842222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.842305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.842390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.842480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.842560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.842637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.842716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.842793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.842872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.842950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.843027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.843103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.843183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.843260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.843337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.843413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.843500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.843579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.843658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.843738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.843815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.843893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.843970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.844047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.844125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.844205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.844286] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.844396] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.844400] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40551, lrc_seqno=40551, guc_id=0, flags=0x73 in no process [-1] <7> [297.844403] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.845315] ------------[ cut here ]------------ <4> [297.845317] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.845318] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.845391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.845459] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.845470] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.845472] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.845474] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.845475] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.845480] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.845549] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.845551] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.845553] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.845554] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.845556] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.845557] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.845558] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.845559] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.845561] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.845562] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.845564] PKRU: 55555554 <4> [297.845565] Call Trace: <4> [297.845566] <TASK> <4> [297.845570] ? lock_sync+0x100/0x100 <4> [297.845575] ? lock_release+0xd0/0x2b0 <4> [297.845580] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.845586] process_one_work+0x239/0x760 <4> [297.845592] worker_thread+0x200/0x3f0 <4> [297.845595] ? __pfx_worker_thread+0x10/0x10 <4> [297.845597] kthread+0x10d/0x150 <4> [297.845600] ? __pfx_kthread+0x10/0x10 <4> [297.845604] ret_from_fork+0x3d4/0x480 <4> [297.845606] ? __pfx_kthread+0x10/0x10 <4> [297.845609] ret_from_fork_asm+0x1a/0x30 <4> [297.845617] </TASK> <4> [297.845618] irq event stamp: 1007097 <4> [297.845619] hardirqs last enabled at (1007103): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.845622] hardirqs last disabled at (1007108): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.845624] softirqs last enabled at (1006058): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.845627] softirqs last disabled at (1006047): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.845629] ---[ end trace 0000000000000000 ]--- <6> [297.845631] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.845701] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.845708] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.846115] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.846220] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.846326] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.846531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.846620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.846712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.846803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.846891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.846978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.847065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.847152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.847235] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.847330] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.847451] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.848457] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.858431] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.858677] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.859834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.859914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.859984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.860055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.860126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.860195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.860264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.860333] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.860401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.860485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.860560] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.860638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.860715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.860792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.860867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.860942] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.861017] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.861091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.861165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.861245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.861326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.861406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.861497] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.861577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.861655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.861737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.861818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.861900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.861982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.862062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.862141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.862221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.862306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.862391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.862489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.862573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.862655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.862734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.862811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.862888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.862968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.863047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.863125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.863200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.863279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.863356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.863436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.863512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.863589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.863665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.863745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.863829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.863911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.863992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.864070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.864151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.864228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.864304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.864387] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.864506] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.864510] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40551, lrc_seqno=40551, guc_id=0, flags=0x73 in no process [-1] <7> [297.864512] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.864572] ------------[ cut here ]------------ <4> [297.864573] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.864574] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.864646] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.864710] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.864718] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.864720] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.864721] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.864723] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.864728] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.864799] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.864800] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.864802] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.864804] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.864805] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.864806] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.864807] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.864809] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.864810] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.864812] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.864813] PKRU: 55555554 <4> [297.864814] Call Trace: <4> [297.864815] <TASK> <4> [297.864819] ? lock_sync+0x100/0x100 <4> [297.864824] ? lock_release+0xd0/0x2b0 <4> [297.864829] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.864834] process_one_work+0x239/0x760 <4> [297.864841] worker_thread+0x200/0x3f0 <4> [297.864844] ? __pfx_worker_thread+0x10/0x10 <4> [297.864846] kthread+0x10d/0x150 <4> [297.864849] ? __pfx_kthread+0x10/0x10 <4> [297.864853] ret_from_fork+0x3d4/0x480 <4> [297.864855] ? __pfx_kthread+0x10/0x10 <4> [297.864859] ret_from_fork_asm+0x1a/0x30 <4> [297.864867] </TASK> <4> [297.864868] irq event stamp: 1010211 <4> [297.864869] hardirqs last enabled at (1010217): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.864872] hardirqs last disabled at (1010222): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.864875] softirqs last enabled at (1009260): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.864877] softirqs last disabled at (1009253): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.864879] ---[ end trace 0000000000000000 ]--- <5> [297.865255] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40606, lrc_seqno=40606, guc_id=0, flags=0x73 in no process [-1] <7> [297.865261] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.865639] ------------[ cut here ]------------ <4> [297.865641] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.865643] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [297.865769] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.865841] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.865850] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.865855] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.865856] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.865858] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.865866] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.865940] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.865943] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.865946] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.865947] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.865948] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.865950] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.865951] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.865952] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [297.865954] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.865955] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.865957] PKRU: 55555554 <4> [297.865958] Call Trace: <4> [297.865959] <TASK> <4> [297.865964] ? lock_sync+0x100/0x100 <4> [297.865969] ? lock_release+0xd0/0x2b0 <4> [297.865974] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.865981] process_one_work+0x239/0x760 <4> [297.865988] worker_thread+0x200/0x3f0 <4> [297.865991] ? __pfx_worker_thread+0x10/0x10 <4> [297.865993] kthread+0x10d/0x150 <4> [297.865997] ? __pfx_kthread+0x10/0x10 <4> [297.866000] ret_from_fork+0x3d4/0x480 <4> [297.866003] ? __pfx_kthread+0x10/0x10 <4> [297.866006] ret_from_fork_asm+0x1a/0x30 <4> [297.866013] </TASK> <4> [297.866014] irq event stamp: 1858835 <4> [297.866015] hardirqs last enabled at (1858841): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.866019] hardirqs last disabled at (1858846): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.866022] softirqs last enabled at (1858040): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.866025] softirqs last disabled at (1858031): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.866027] ---[ end trace 0000000000000000 ]--- <6> [297.866030] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.866111] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.866220] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.866330] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.866442] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.866551] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.866756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.866846] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.866956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.867045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.867132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.867217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.867302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.867389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.867480] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.867574] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.867690] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.868696] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.878431] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.878682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.879671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.879751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.879829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.879906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.879983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.880058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.880133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.880208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.880282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.880357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.880433] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.880508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.880582] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.880659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.880737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.880814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.880890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.880966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.881041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.881124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.881205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.881284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.881366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.881449] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.881528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.881610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.881691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.881774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.881856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.881936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.882015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.882095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.882181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.882267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.882351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.882436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.882516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.882594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.882672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.882749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.882833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.882915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.882996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.883073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.883154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.883232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.883310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.883387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.883473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.883549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.883626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.883704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.883780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.883857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.883937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.884018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.884097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.884175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.884256] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.884368] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.884373] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40606, lrc_seqno=40606, guc_id=0, flags=0x73 in no process [-1] <7> [297.884375] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.884440] ------------[ cut here ]------------ <4> [297.884441] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.884443] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [297.884514] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.884578] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.884586] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.884589] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.884590] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.884592] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.884597] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.884667] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.884669] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.884671] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.884673] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.884674] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.884675] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.884677] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.884678] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [297.884679] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.884681] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.884682] PKRU: 55555554 <4> [297.884684] Call Trace: <4> [297.884685] <TASK> <4> [297.884689] ? lock_sync+0x100/0x100 <4> [297.884693] ? lock_release+0xd0/0x2b0 <4> [297.884698] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.884704] process_one_work+0x239/0x760 <4> [297.884710] worker_thread+0x200/0x3f0 <4> [297.884713] ? __pfx_worker_thread+0x10/0x10 <4> [297.884716] kthread+0x10d/0x150 <4> [297.884719] ? __pfx_kthread+0x10/0x10 <4> [297.884722] ret_from_fork+0x3d4/0x480 <4> [297.884724] ? __pfx_kthread+0x10/0x10 <4> [297.884728] ret_from_fork_asm+0x1a/0x30 <4> [297.884735] </TASK> <4> [297.884736] irq event stamp: 1862053 <4> [297.884737] hardirqs last enabled at (1862059): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.884740] hardirqs last disabled at (1862064): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.884743] softirqs last enabled at (1861020): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.884745] softirqs last disabled at (1861013): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.884747] ---[ end trace 0000000000000000 ]--- <6> [297.884749] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.884819] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.884826] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.885158] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.885361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.885453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.885547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.885634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.885722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.885808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.885894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.885982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.886067] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.886163] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.886280] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.887285] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.898007] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.898248] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.899298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.899369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.899442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.899518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.899596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.899675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.899754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.899830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.899905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.899980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.900055] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.900129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.900202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.900277] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.900352] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.900430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.900504] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.900578] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.900651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.900733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.900814] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.900893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.900974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.901051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.901132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.901212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.901292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.901373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.901461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.901541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.901620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.901699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.901787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.901875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.901962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.902045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.902125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.902201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.902278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.902354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.902437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.902515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.902597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.902679] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.902762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.902841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.902919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.902996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.903074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.903149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.903225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.903303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.903380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.903465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.903541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.903619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.903698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.903776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.903857] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.903966] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.903970] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40606, lrc_seqno=40606, guc_id=0, flags=0x73 in no process [-1] <7> [297.903972] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.904031] ------------[ cut here ]------------ <4> [297.904032] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.904033] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [297.904104] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.904167] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.904175] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.904178] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.904179] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.904181] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.904186] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.904254] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.904256] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.904258] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.904260] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.904261] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.904262] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.904263] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.904264] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [297.904266] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.904267] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.904269] PKRU: 55555554 <4> [297.904270] Call Trace: <4> [297.904271] <TASK> <4> [297.904275] ? lock_sync+0x100/0x100 <4> [297.904279] ? lock_release+0xd0/0x2b0 <4> [297.904285] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.904290] process_one_work+0x239/0x760 <4> [297.904296] worker_thread+0x200/0x3f0 <4> [297.904299] ? __pfx_worker_thread+0x10/0x10 <4> [297.904301] kthread+0x10d/0x150 <4> [297.904304] ? __pfx_kthread+0x10/0x10 <4> [297.904308] ret_from_fork+0x3d4/0x480 <4> [297.904310] ? __pfx_kthread+0x10/0x10 <4> [297.904313] ret_from_fork_asm+0x1a/0x30 <4> [297.904320] </TASK> <4> [297.904322] irq event stamp: 1865151 <4> [297.904323] hardirqs last enabled at (1865157): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.904325] hardirqs last disabled at (1865162): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.904328] softirqs last enabled at (1864248): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.904330] softirqs last disabled at (1864241): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.904332] ---[ end trace 0000000000000000 ]--- <5> [297.906058] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40661, lrc_seqno=40661, guc_id=0, flags=0x73 in no process [-1] <7> [297.906061] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [297.906069] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [297.906123] ------------[ cut here ]------------ <4> [297.906124] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.906125] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.906196] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic <7> [297.906173] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [297.906222] cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.906267] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.906269] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.906271] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.906273] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.906278] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.906347] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.906348] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.906350] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.906351] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.906353] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.906354] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.906355] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.906356] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.906358] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.906359] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.906361] PKRU: 55555554 <4> [297.906362] Call Trace: <4> [297.906363] <TASK> <4> [297.906367] ? lock_sync+0x100/0x100 <4> [297.906371] ? lock_release+0xd0/0x2b0 <4> [297.906376] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.906382] process_one_work+0x239/0x760 <4> [297.906388] worker_thread+0x200/0x3f0 <4> [297.906390] ? __pfx_worker_thread+0x10/0x10 <4> [297.906393] kthread+0x10d/0x150 <4> [297.906396] ? __pfx_kthread+0x10/0x10 <4> [297.906399] ret_from_fork+0x3d4/0x480 <4> [297.906401] ? __pfx_kthread+0x10/0x10 <4> [297.906404] ret_from_fork_asm+0x1a/0x30 <4> [297.906412] </TASK> <4> [297.906413] irq event stamp: 1013409 <4> [297.906414] hardirqs last enabled at (1013415): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.906417] hardirqs last disabled at (1013420): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.906419] softirqs last enabled at (1009260): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.906421] softirqs last disabled at (1009253): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.906428] ---[ end trace 0000000000000000 ]--- <6> [297.906430] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.906501] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.906508] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.906559] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.906815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.906952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.907092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.907227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.907362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.907502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.907637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.907772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.907903] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.908046] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.908213] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.909621] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.919434] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.919745] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.920888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.921010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.921130] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.921251] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.921372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.921500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.921622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.921744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.921864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.921985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.922106] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.922227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.922348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.922473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.922596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.922717] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.922837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.922958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.923079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.923204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.923329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.923461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.923589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.923714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.923837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.923960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.924088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.924216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.924343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.924476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.924604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.924731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.924865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.924998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.925129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.925258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.925387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.925526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.925621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.925707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.925792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.925875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.925956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.926035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.926117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.926195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.926275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.926352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.926441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.926520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.926599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.926683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.926765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.926844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.926922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.927001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.927079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.927156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.927236] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.927350] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.927355] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40661, lrc_seqno=40661, guc_id=0, flags=0x73 in no process [-1] <7> [297.927357] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.927417] ------------[ cut here ]------------ <4> [297.927418] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.927419] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#0: kworker/u64:1/120 <4> [297.927516] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.927583] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.927591] CPU: 0 UID: 0 PID: 120 Comm: kworker/u64:1 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.927594] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.927595] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.927597] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.927602] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.927674] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.927675] RSP: 0018:ffffc9000055bca0 EFLAGS: 00010246 <4> [297.927678] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.927679] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.927681] RBP: ffffc9000055bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.927682] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.927683] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.927684] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [297.927686] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.927687] CR2: 000072fba006d1c8 CR3: 000000011283f004 CR4: 0000000000f72ef0 <4> [297.927689] PKRU: 55555554 <4> [297.927690] Call Trace: <4> [297.927691] <TASK> <4> [297.927695] ? lock_sync+0x100/0x100 <4> [297.927700] ? lock_release+0xd0/0x2b0 <4> [297.927705] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.927711] process_one_work+0x239/0x760 <4> [297.927717] worker_thread+0x200/0x3f0 <4> [297.927720] ? __pfx_worker_thread+0x10/0x10 <4> [297.927722] kthread+0x10d/0x150 <4> [297.927725] ? __pfx_kthread+0x10/0x10 <4> [297.927729] ret_from_fork+0x3d4/0x480 <4> [297.927731] ? __pfx_kthread+0x10/0x10 <4> [297.927734] ret_from_fork_asm+0x1a/0x30 <4> [297.927741] </TASK> <4> [297.927742] irq event stamp: 1039999 <4> [297.927743] hardirqs last enabled at (1040005): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.927746] hardirqs last disabled at (1040010): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.927749] softirqs last enabled at (1039226): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.927751] softirqs last disabled at (1039221): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.927753] ---[ end trace 0000000000000000 ]--- <6> [297.928683] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.928753] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.928762] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.928779] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.928883] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.928988] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.929193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.929282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.929375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.929473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.929560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.929646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.929732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.929820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.929903] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.929996] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.930113] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.931120] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.941429] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.941676] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.942738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.942818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.942896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.942973] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.943050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.943126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.943201] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.943276] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.943355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.943435] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.943512] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.943587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.943662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.943737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.943812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.943886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.943960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.944034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.944109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.944189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.944269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.944348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.944433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.944517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.944598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.944677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.944754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.944839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.944941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.945034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.945125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.945203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.945288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.945371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.945466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.945552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.945636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.945716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.945794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.945874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.945958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.946039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.946119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.946197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.946278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.946355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.946436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.946512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.946590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.946666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.946742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.946821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.946898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.946978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.947059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.947139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.947216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.947293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.947373] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.947492] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.947496] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40661, lrc_seqno=40661, guc_id=0, flags=0x73 in no process [-1] <7> [297.947499] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.947574] ------------[ cut here ]------------ <4> [297.947576] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.947577] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [297.947648] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.947728] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.947738] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.947741] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.947743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.947744] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.947750] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.947819] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.947821] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [297.947824] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.947826] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.947827] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.947829] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.947830] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.947832] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [297.947834] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.947835] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [297.947837] PKRU: 55555554 <4> [297.947839] Call Trace: <4> [297.947840] <TASK> <4> [297.947844] ? lock_sync+0x100/0x100 <4> [297.947849] ? lock_release+0xd0/0x2b0 <4> [297.947855] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.947861] process_one_work+0x239/0x760 <4> [297.947867] worker_thread+0x200/0x3f0 <4> [297.947870] ? __pfx_worker_thread+0x10/0x10 <4> [297.947873] kthread+0x10d/0x150 <4> [297.947876] ? __pfx_kthread+0x10/0x10 <4> [297.947879] ret_from_fork+0x3d4/0x480 <4> [297.947881] ? __pfx_kthread+0x10/0x10 <4> [297.947885] ret_from_fork_asm+0x1a/0x30 <4> [297.947892] </TASK> <4> [297.947893] irq event stamp: 1869457 <4> [297.947894] hardirqs last enabled at (1869463): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.947897] hardirqs last disabled at (1869468): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.947900] softirqs last enabled at (1868504): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.947902] softirqs last disabled at (1868497): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.947904] ---[ end trace 0000000000000000 ]--- <5> [297.948255] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40716, lrc_seqno=40716, guc_id=0, flags=0x73 in no process [-1] <7> [297.948258] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.948527] ------------[ cut here ]------------ <4> [297.948529] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.948531] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.948605] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.948681] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.948690] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.948693] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.948694] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.948696] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.948703] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.948778] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.948779] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.948781] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.948782] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.948784] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.948785] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.948786] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.948788] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.948789] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.948790] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.948792] PKRU: 55555554 <4> [297.948793] Call Trace: <4> [297.948794] <TASK> <4> [297.948799] ? lock_sync+0x100/0x100 <4> [297.948803] ? lock_release+0xd0/0x2b0 <4> [297.948809] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.948816] process_one_work+0x239/0x760 <4> [297.948821] worker_thread+0x200/0x3f0 <4> [297.948824] ? __pfx_worker_thread+0x10/0x10 <4> [297.948826] kthread+0x10d/0x150 <4> [297.948829] ? __pfx_kthread+0x10/0x10 <4> [297.948833] ret_from_fork+0x3d4/0x480 <4> [297.948835] ? __pfx_kthread+0x10/0x10 <4> [297.948838] ret_from_fork_asm+0x1a/0x30 <4> [297.948846] </TASK> <4> [297.948848] irq event stamp: 1015547 <4> [297.948849] hardirqs last enabled at (1015553): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.948852] hardirqs last disabled at (1015558): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.948854] softirqs last enabled at (1014756): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.948857] softirqs last disabled at (1014749): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.948859] ---[ end trace 0000000000000000 ]--- <6> [297.948862] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.948937] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.948953] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.949403] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.949520] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.949627] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.949829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.949918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.950013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.950105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.950196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.950286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.950376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.950473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.950556] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.950653] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.950770] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.951779] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.962765] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.963015] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.963992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.964070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.964145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.964222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.964298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.964374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.964458] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.964534] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.964611] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.964690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.964768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.964845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.964921] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.964996] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.965071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.965144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.965217] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.965292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.965366] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.965451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.965532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.965615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.965698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.965777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.965854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.965934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.966013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.966094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.966175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.966255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.966334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.966411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.966508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.966592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.966676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.966758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.966841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.966922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.967001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.967078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.967158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.967237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.967315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.967393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.967482] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.967560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.967641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.967722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.967803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.967880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.967957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.968037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.968113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.968191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.968267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.968343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.968418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.968506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.968585] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.968696] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.968700] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40716, lrc_seqno=40716, guc_id=0, flags=0x73 in no process [-1] <7> [297.968702] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.968760] ------------[ cut here ]------------ <4> [297.968761] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.968763] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#4: kworker/u64:14/2466 <4> [297.968832] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.968896] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.968904] CPU: 4 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.968907] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.968908] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.968909] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.968914] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.968982] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.968984] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [297.968986] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.968987] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.968988] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.968990] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.968991] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.968992] FS: 0000000000000000(0000) GS:ffff8888dae97000(0000) knlGS:0000000000000000 <4> [297.968994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.968995] CR2: 000072fba006c418 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.968997] PKRU: 55555554 <4> [297.968998] Call Trace: <4> [297.968999] <TASK> <4> [297.969003] ? lock_sync+0x100/0x100 <4> [297.969007] ? lock_release+0xd0/0x2b0 <4> [297.969013] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.969018] process_one_work+0x239/0x760 <4> [297.969025] worker_thread+0x200/0x3f0 <4> [297.969027] ? __pfx_worker_thread+0x10/0x10 <4> [297.969030] kthread+0x10d/0x150 <4> [297.969032] ? __pfx_kthread+0x10/0x10 <4> [297.969036] ret_from_fork+0x3d4/0x480 <4> [297.969038] ? __pfx_kthread+0x10/0x10 <4> [297.969042] ret_from_fork_asm+0x1a/0x30 <4> [297.969049] </TASK> <4> [297.969050] irq event stamp: 1018653 <4> [297.969051] hardirqs last enabled at (1018659): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.969054] hardirqs last disabled at (1018664): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.969056] softirqs last enabled at (1017686): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.969059] softirqs last disabled at (1017679): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.969061] ---[ end trace 0000000000000000 ]--- <6> [297.969063] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.969131] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.969137] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.970372] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.970487] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.970593] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.970804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.970897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.970994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.971086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.971174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.971260] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.971345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.971442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.971533] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.971631] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.971748] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.972762] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [297.983767] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [297.984010] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [297.985228] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [297.985360] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [297.985493] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [297.985616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [297.985738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [297.985859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [297.985980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [297.986100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [297.986221] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [297.986342] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [297.986468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [297.986590] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [297.986711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [297.986832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [297.986953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [297.987073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [297.987194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [297.987315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [297.987439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [297.987567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [297.987693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [297.987817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [297.987943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [297.988066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [297.988190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [297.988313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [297.988441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [297.988569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [297.988697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [297.988822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [297.988949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [297.989075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [297.989207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [297.989339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [297.989474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [297.989604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [297.989732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [297.989857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [297.989982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [297.990105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [297.990233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [297.990359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [297.990488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [297.990612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [297.990741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [297.990866] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [297.990991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [297.991115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [297.991240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [297.991363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [297.991492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [297.991620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [297.991745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [297.991869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [297.991993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [297.992119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [297.992242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [297.992365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [297.992500] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [297.992664] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [297.992672] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40716, lrc_seqno=40716, guc_id=0, flags=0x73 in no process [-1] <7> [297.992675] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.992777] ------------[ cut here ]------------ <4> [297.992779] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.992781] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:13/2465 <4> [297.992894] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.992986] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.992998] CPU: 12 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.993002] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.993003] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.993005] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.993013] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.993125] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.993127] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [297.993130] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.993133] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.993134] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.993136] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.993138] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.993140] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [297.993142] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.993144] CR2: 000072fba006e408 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.993146] PKRU: 55555554 <4> [297.993148] Call Trace: <4> [297.993150] <TASK> <4> [297.993156] ? lock_sync+0x100/0x100 <4> [297.993164] ? lock_release+0xd0/0x2b0 <4> [297.993172] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.993181] process_one_work+0x239/0x760 <4> [297.993192] worker_thread+0x200/0x3f0 <4> [297.993196] ? __pfx_worker_thread+0x10/0x10 <4> [297.993199] kthread+0x10d/0x150 <4> [297.993203] ? __pfx_kthread+0x10/0x10 <4> [297.993208] ret_from_fork+0x3d4/0x480 <4> [297.993211] ? __pfx_kthread+0x10/0x10 <4> [297.993216] ret_from_fork_asm+0x1a/0x30 <4> [297.993228] </TASK> <4> [297.993229] irq event stamp: 817781 <4> [297.993231] hardirqs last enabled at (817787): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.993235] hardirqs last disabled at (817792): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.993238] softirqs last enabled at (816908): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.993241] softirqs last disabled at (816899): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.993244] ---[ end trace 0000000000000000 ]--- <5> [297.993730] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40771, lrc_seqno=40771, guc_id=0, flags=0x73 in no process [-1] <7> [297.993734] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [297.993836] ------------[ cut here ]------------ <4> [297.993837] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [297.993839] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:13/2465 <4> [297.993951] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [297.994039] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [297.994050] CPU: 12 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [297.994053] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [297.994055] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [297.994057] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [297.994064] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [297.994175] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [297.994177] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [297.994180] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [297.994182] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [297.994184] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [297.994186] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [297.994187] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [297.994189] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [297.994191] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [297.994193] CR2: 000072fba006e408 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [297.994195] PKRU: 55555554 <4> [297.994197] Call Trace: <4> [297.994198] <TASK> <4> [297.994204] ? lock_sync+0x100/0x100 <4> [297.994210] ? lock_release+0xd0/0x2b0 <4> [297.994219] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [297.994228] process_one_work+0x239/0x760 <4> [297.994237] worker_thread+0x200/0x3f0 <4> [297.994241] ? __pfx_worker_thread+0x10/0x10 <4> [297.994244] kthread+0x10d/0x150 <4> [297.994248] ? __pfx_kthread+0x10/0x10 <4> [297.994253] ret_from_fork+0x3d4/0x480 <4> [297.994256] ? __pfx_kthread+0x10/0x10 <4> [297.994261] ret_from_fork_asm+0x1a/0x30 <4> [297.994273] </TASK> <4> [297.994274] irq event stamp: 819479 <4> [297.994276] hardirqs last enabled at (819485): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [297.994279] hardirqs last disabled at (819490): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [297.994282] softirqs last enabled at (816908): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.994285] softirqs last disabled at (816899): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [297.994288] ---[ end trace 0000000000000000 ]--- <6> [297.994290] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [297.994403] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [297.994410] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [297.995895] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [297.996064] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [297.996225] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [297.996489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [297.996621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [297.996758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [297.996890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [297.997022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [297.997153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [297.997285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [297.997424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [297.997605] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [297.997722] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [297.997848] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [297.998867] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.009479] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.009746] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.010880] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.010968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.011052] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.011136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.011219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.011303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.011386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.011476] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.011556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.011635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.011714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.011792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.011871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.011949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.012029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.012109] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.012192] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.012275] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.012356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.012447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.012534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.012619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.012707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.012792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.012876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.012959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.013041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.013126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.013210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.013295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.013379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.013473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.013568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.013660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.013750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.013836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.013921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.014003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.014084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.014165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.014249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.014332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.014416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.014513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.014602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.014686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.014769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.014851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.014934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.015015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.015096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.015181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.015263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.015348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.015437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.015523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.015606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.015689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.015775] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.015892] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.015897] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40771, lrc_seqno=40771, guc_id=0, flags=0x73 in no process [-1] <7> [298.015900] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.015962] ------------[ cut here ]------------ <4> [298.015963] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.015965] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.016041] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.016106] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.016114] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.016117] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.016119] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.016120] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.016126] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.016201] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.016202] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.016205] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.016206] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.016207] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.016209] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.016210] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.016211] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.016213] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.016214] CR2: 00007f74cf191008 CR3: 00000001d79d2001 CR4: 0000000000f72ef0 <4> [298.016216] PKRU: 55555554 <4> [298.016217] Call Trace: <4> [298.016218] <TASK> <4> [298.016222] ? lock_sync+0x100/0x100 <4> [298.016227] ? lock_release+0xd0/0x2b0 <4> [298.016232] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.016238] process_one_work+0x239/0x760 <4> [298.016245] worker_thread+0x200/0x3f0 <4> [298.016248] ? __pfx_worker_thread+0x10/0x10 <4> [298.016250] kthread+0x10d/0x150 <4> [298.016253] ? __pfx_kthread+0x10/0x10 <4> [298.016257] ret_from_fork+0x3d4/0x480 <4> [298.016259] ? __pfx_kthread+0x10/0x10 <4> [298.016263] ret_from_fork_asm+0x1a/0x30 <4> [298.016270] </TASK> <4> [298.016271] irq event stamp: 822557 <4> [298.016272] hardirqs last enabled at (822563): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.016275] hardirqs last disabled at (822568): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.016277] softirqs last enabled at (821486): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.016280] softirqs last disabled at (821481): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.016282] ---[ end trace 0000000000000000 ]--- <6> [298.016284] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.016358] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.016364] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.017691] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.017800] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.017912] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.018120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.018216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.018314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.018408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.018513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.018607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.018701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.018795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.018884] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.018986] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.019109] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.020134] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.030428] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.030682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.031838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.031924] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.032006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.032089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.032170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.032253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.032337] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.032419] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.032517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.032597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.032676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.032754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.032833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.032912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.032992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.033070] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.033148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.033227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.033307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.033394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.033491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.033575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.033659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.033740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.033822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.033903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.033986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.034074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.034161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.034245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.034329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.034412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.034513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.034602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.034691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.034777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.034861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.034944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.035027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.035107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.035192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.035278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.035365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.035454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.035542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.035624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.035706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.035787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.035870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.035956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.036039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.036126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.036215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.036303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.036385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.036478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.036561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.036642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.036728] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.036843] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.036848] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40771, lrc_seqno=40771, guc_id=0, flags=0x73 in no process [-1] <7> [298.036850] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.036913] ------------[ cut here ]------------ <4> [298.036914] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.036915] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.036991] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.037055] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.037064] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.037067] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.037068] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.037069] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.037074] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.037149] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.037151] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.037153] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.037155] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.037156] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.037157] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.037158] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.037160] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.037161] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.037163] CR2: 00007f74cf191008 CR3: 00000001d79d2001 CR4: 0000000000f72ef0 <4> [298.037164] PKRU: 55555554 <4> [298.037165] Call Trace: <4> [298.037167] <TASK> <4> [298.037170] ? lock_sync+0x100/0x100 <4> [298.037175] ? lock_release+0xd0/0x2b0 <4> [298.037180] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.037186] process_one_work+0x239/0x760 <4> [298.037192] worker_thread+0x200/0x3f0 <4> [298.037195] ? __pfx_worker_thread+0x10/0x10 <4> [298.037197] kthread+0x10d/0x150 <4> [298.037200] ? __pfx_kthread+0x10/0x10 <4> [298.037204] ret_from_fork+0x3d4/0x480 <4> [298.037206] ? __pfx_kthread+0x10/0x10 <4> [298.037209] ret_from_fork_asm+0x1a/0x30 <4> [298.037217] </TASK> <4> [298.037218] irq event stamp: 825703 <4> [298.037219] hardirqs last enabled at (825709): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.037222] hardirqs last disabled at (825714): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.037224] softirqs last enabled at (824812): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.037227] softirqs last disabled at (824801): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.037229] ---[ end trace 0000000000000000 ]--- <5> [298.038414] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40826, lrc_seqno=40826, guc_id=0, flags=0x73 in no process [-1] <7> [298.038417] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [298.038514] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.038623] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [298.038728] ------------[ cut here ]------------ <4> [298.038729] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.038730] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.038808] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.038871] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.038879] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.038881] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.038883] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.038884] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.038889] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.038964] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.038965] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.038967] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.038969] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.038970] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.038971] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.038972] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.038974] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.038975] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.038976] CR2: 00007f74cf191008 CR3: 00000001d79d2001 CR4: 0000000000f72ef0 <4> [298.038978] PKRU: 55555554 <4> [298.038979] Call Trace: <4> [298.038980] <TASK> <4> [298.038984] ? lock_sync+0x100/0x100 <4> [298.038988] ? lock_release+0xd0/0x2b0 <4> [298.038994] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.039000] process_one_work+0x239/0x760 <4> [298.039006] worker_thread+0x200/0x3f0 <4> [298.039009] ? __pfx_worker_thread+0x10/0x10 <4> [298.039011] kthread+0x10d/0x150 <4> [298.039014] ? __pfx_kthread+0x10/0x10 <4> [298.039018] ret_from_fork+0x3d4/0x480 <4> [298.039020] ? __pfx_kthread+0x10/0x10 <4> [298.039023] ret_from_fork_asm+0x1a/0x30 <4> [298.039030] </TASK> <4> [298.039032] irq event stamp: 827667 <4> [298.039033] hardirqs last enabled at (827673): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.039035] hardirqs last disabled at (827678): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.039038] softirqs last enabled at (826872): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.039040] softirqs last disabled at (826865): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.039042] ---[ end trace 0000000000000000 ]--- <6> [298.039044] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.039118] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.039124] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.039337] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.039547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.039642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.039739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.039835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.039930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.040022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.040114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.040207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.040294] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.040395] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.040525] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.041535] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.051427] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.051682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.052646] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.052733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.052815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.052898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.052980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.053063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.053144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.053224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.053305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.053385] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.053473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.053552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.053631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.053714] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.053795] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.053878] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.053958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.054037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.054117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.054203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.054291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.054376] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.054470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.054553] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.054638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.054721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.054804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.054893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.054982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.055066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.055151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.055236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.055327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.055416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.055518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.055608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.055698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.055784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.055869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.055951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.056037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.056121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.056205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.056287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.056375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.056469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.056554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.056638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.056722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.056805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.056886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.056971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.057053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.057140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.057227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.057314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.057398] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.057489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.057576] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.057691] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.057695] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40826, lrc_seqno=40826, guc_id=0, flags=0x73 in no process [-1] <7> [298.057697] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.057759] ------------[ cut here ]------------ <4> [298.057760] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.057761] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.057839] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.057902] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.057909] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.057912] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.057913] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.057915] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.057919] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.057994] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.057996] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.057998] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.057999] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.058001] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.058002] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.058003] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.058004] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.058006] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.058007] CR2: 00007f74cf191008 CR3: 00000001d79d2001 CR4: 0000000000f72ef0 <4> [298.058009] PKRU: 55555554 <4> [298.058010] Call Trace: <4> [298.058011] <TASK> <4> [298.058015] ? lock_sync+0x100/0x100 <4> [298.058019] ? lock_release+0xd0/0x2b0 <4> [298.058025] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.058031] process_one_work+0x239/0x760 <4> [298.058037] worker_thread+0x200/0x3f0 <4> [298.058039] ? __pfx_worker_thread+0x10/0x10 <4> [298.058042] kthread+0x10d/0x150 <4> [298.058045] ? __pfx_kthread+0x10/0x10 <4> [298.058048] ret_from_fork+0x3d4/0x480 <4> [298.058050] ? __pfx_kthread+0x10/0x10 <4> [298.058054] ret_from_fork_asm+0x1a/0x30 <4> [298.058061] </TASK> <4> [298.058062] irq event stamp: 830757 <4> [298.058063] hardirqs last enabled at (830763): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.058066] hardirqs last disabled at (830768): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.058068] softirqs last enabled at (829798): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.058071] softirqs last disabled at (829791): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.058073] ---[ end trace 0000000000000000 ]--- <6> [298.058075] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.058150] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.058155] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.059386] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.059508] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.059619] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.059829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.059921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.060015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.060105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.060192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.060279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.060368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.060464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.060546] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.060641] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.060757] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.061765] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.072427] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.072677] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.073750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.073830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.073908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.073987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.074063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.074138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.074213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.074290] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.074368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.074451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.074526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.074601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.074676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.074750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.074824] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.074898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.074972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.075049] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.075127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.075211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.075292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.075371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.075463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.075543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.075622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.075699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.075796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.075879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.075962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.076044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.076125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.076206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.076292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.076377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.076475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.076560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.076642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.076721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.076798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.076874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.076954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.077032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.077109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.077186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.077266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.077343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.077429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.077505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.077583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.077659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.077736] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.077815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.077892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.077969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.078045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.078123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.078199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.078280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.078362] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.078478] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.078482] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40826, lrc_seqno=40826, guc_id=0, flags=0x73 in no process [-1] <7> [298.078485] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.078544] ------------[ cut here ]------------ <4> [298.078545] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.078547] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.078618] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.078683] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.078692] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.078695] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.078696] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.078697] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.078703] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.078773] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.078774] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.078777] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.078778] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.078779] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.078781] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.078782] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.078783] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.078785] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.078786] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.078788] PKRU: 55555554 <4> [298.078789] Call Trace: <4> [298.078790] <TASK> <4> [298.078794] ? lock_sync+0x100/0x100 <4> [298.078799] ? lock_release+0xd0/0x2b0 <4> [298.078804] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.078810] process_one_work+0x239/0x760 <4> [298.078816] worker_thread+0x200/0x3f0 <4> [298.078819] ? __pfx_worker_thread+0x10/0x10 <4> [298.078821] kthread+0x10d/0x150 <4> [298.078824] ? __pfx_kthread+0x10/0x10 <4> [298.078828] ret_from_fork+0x3d4/0x480 <4> [298.078830] ? __pfx_kthread+0x10/0x10 <4> [298.078833] ret_from_fork_asm+0x1a/0x30 <4> [298.078841] </TASK> <4> [298.078842] irq event stamp: 1874021 <4> [298.078843] hardirqs last enabled at (1874027): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.078846] hardirqs last disabled at (1874032): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.078848] softirqs last enabled at (1873072): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.078851] softirqs last disabled at (1873031): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.078853] ---[ end trace 0000000000000000 ]--- <5> [298.080228] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40881, lrc_seqno=40881, guc_id=0, flags=0x73 in no process [-1] <7> [298.080230] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.080291] ------------[ cut here ]------------ <4> [298.080292] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.080293] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.080362] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.080429] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.080438] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.080441] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.080442] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.080443] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.080448] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.080517] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.080518] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.080520] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.080522] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.080523] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.080524] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.080525] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.080526] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.080528] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.080529] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.080531] PKRU: 55555554 <4> [298.080532] Call Trace: <4> [298.080533] <TASK> <4> [298.080536] ? lock_sync+0x100/0x100 <4> [298.080541] ? lock_release+0xd0/0x2b0 <4> [298.080546] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.080552] process_one_work+0x239/0x760 <4> [298.080557] worker_thread+0x200/0x3f0 <4> [298.080560] ? __pfx_worker_thread+0x10/0x10 <4> [298.080562] kthread+0x10d/0x150 <4> [298.080565] ? __pfx_kthread+0x10/0x10 <4> [298.080569] ret_from_fork+0x3d4/0x480 <4> [298.080571] ? __pfx_kthread+0x10/0x10 <4> [298.080574] ret_from_fork_asm+0x1a/0x30 <4> [298.080582] </TASK> <4> [298.080583] irq event stamp: 1875959 <4> [298.080584] hardirqs last enabled at (1875965): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.080586] hardirqs last disabled at (1875970): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.080589] softirqs last enabled at (1875654): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.080591] softirqs last disabled at (1875649): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.080593] ---[ end trace 0000000000000000 ]--- <6> [298.080595] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.080664] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.080669] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.080938] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.081040] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.081145] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.081349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.081440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.081530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.081620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.081708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.081794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.081880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.081967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.082048] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.082143] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.082260] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.083268] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.093426] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.093676] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.094697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.094777] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.094855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.094932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.095008] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.095082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.095156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.095231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.095306] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.095380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.095464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.095538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.095614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.095691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.095768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.095843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.095918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.095993] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.096066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.096147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.096228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.096308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.096391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.096478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.096556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.096633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.096711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.096790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.096869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.096948] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.097026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.097108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.097195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.097282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.097368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.097454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.097535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.097613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.097695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.097775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.097857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.097936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.098014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.098091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.098171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.098248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.098325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.098401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.098493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.098574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.098652] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.098732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.098810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.098891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.098971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.099051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.099126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.099202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.099283] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.099395] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.099399] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40881, lrc_seqno=40881, guc_id=0, flags=0x73 in no process [-1] <7> [298.099402] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.100321] ------------[ cut here ]------------ <4> [298.100322] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.100323] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.100397] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.100465] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.100473] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.100476] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.100477] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.100479] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.100484] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.100553] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.100555] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.100557] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.100559] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.100560] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.100561] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.100562] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.100563] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.100565] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.100566] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.100568] PKRU: 55555554 <4> [298.100569] Call Trace: <4> [298.100570] <TASK> <4> [298.100574] ? lock_sync+0x100/0x100 <4> [298.100578] ? lock_release+0xd0/0x2b0 <4> [298.100584] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.100589] process_one_work+0x239/0x760 <4> [298.100595] worker_thread+0x200/0x3f0 <4> [298.100598] ? __pfx_worker_thread+0x10/0x10 <4> [298.100601] kthread+0x10d/0x150 <4> [298.100603] ? __pfx_kthread+0x10/0x10 <4> [298.100607] ret_from_fork+0x3d4/0x480 <4> [298.100609] ? __pfx_kthread+0x10/0x10 <4> [298.100612] ret_from_fork_asm+0x1a/0x30 <4> [298.100620] </TASK> <4> [298.100621] irq event stamp: 1879053 <4> [298.100622] hardirqs last enabled at (1879059): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.100625] hardirqs last disabled at (1879064): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.100627] softirqs last enabled at (1877920): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.100629] softirqs last disabled at (1877913): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.100631] ---[ end trace 0000000000000000 ]--- <6> [298.100633] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.100704] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.100710] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.101126] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.101230] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.101334] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.101536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.101623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.101716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.101805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.101892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.101979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.102065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.102153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.102239] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.102333] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.102456] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.103464] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.113425] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.113674] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.114797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.114877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.114950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.115021] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.115091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.115161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.115232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.115304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.115375] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.115452] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.115538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.115614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.115688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.115764] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.115841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.115917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.115991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.116066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.116139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.116221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.116305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.116387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.116487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.116566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.116643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.116720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.116798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.116881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.116962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.117042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.117120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.117198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.117282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.117365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.117454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.117538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.117620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.117700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.117779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.117857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.117942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.118024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.118103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.118181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.118262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.118338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.118415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.118510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.118594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.118675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.118753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.118835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.118915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.118993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.119071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.119150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.119227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.119304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.119387] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.119510] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.119514] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40881, lrc_seqno=40881, guc_id=0, flags=0x73 in no process [-1] <7> [298.119516] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.119576] ------------[ cut here ]------------ <4> [298.119577] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.119578] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.119651] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.119715] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.119724] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.119727] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.119728] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.119729] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.119734] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.119805] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.119807] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.119809] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.119810] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.119812] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.119813] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.119814] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.119815] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.119817] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.119818] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.119820] PKRU: 55555554 <4> [298.119821] Call Trace: <4> [298.119822] <TASK> <4> [298.119826] ? lock_sync+0x100/0x100 <4> [298.119831] ? lock_release+0xd0/0x2b0 <4> [298.119836] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.119842] process_one_work+0x239/0x760 <4> [298.119848] worker_thread+0x200/0x3f0 <4> [298.119851] ? __pfx_worker_thread+0x10/0x10 <4> [298.119853] kthread+0x10d/0x150 <4> [298.119856] ? __pfx_kthread+0x10/0x10 <4> [298.119859] ret_from_fork+0x3d4/0x480 <4> [298.119861] ? __pfx_kthread+0x10/0x10 <4> [298.119865] ret_from_fork_asm+0x1a/0x30 <4> [298.119872] </TASK> <4> [298.119873] irq event stamp: 1882175 <4> [298.119874] hardirqs last enabled at (1882181): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.119877] hardirqs last disabled at (1882186): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.119879] softirqs last enabled at (1881310): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.119882] softirqs last disabled at (1881303): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.119884] ---[ end trace 0000000000000000 ]--- <7> [298.121536] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.121644] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [298.121746] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40936, lrc_seqno=40936, guc_id=0, flags=0x73 in no process [-1] <7> [298.121748] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.121808] ------------[ cut here ]------------ <4> [298.121809] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.121810] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.121882] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.121946] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.121954] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.121957] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.121958] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.121959] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.121964] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.122035] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.122037] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.122039] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.122040] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.122041] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.122043] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.122044] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.122045] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.122047] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.122048] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.122049] PKRU: 55555554 <4> [298.122051] Call Trace: <4> [298.122052] <TASK> <4> [298.122055] ? lock_sync+0x100/0x100 <4> [298.122060] ? lock_release+0xd0/0x2b0 <4> [298.122065] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.122071] process_one_work+0x239/0x760 <4> [298.122077] worker_thread+0x200/0x3f0 <4> [298.122079] ? __pfx_worker_thread+0x10/0x10 <4> [298.122082] kthread+0x10d/0x150 <4> [298.122084] ? __pfx_kthread+0x10/0x10 <4> [298.122088] ret_from_fork+0x3d4/0x480 <4> [298.122090] ? __pfx_kthread+0x10/0x10 <4> [298.122093] ret_from_fork_asm+0x1a/0x30 <4> [298.122101] </TASK> <4> [298.122102] irq event stamp: 1884073 <4> [298.122103] hardirqs last enabled at (1884079): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.122106] hardirqs last disabled at (1884084): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.122108] softirqs last enabled at (1881310): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.122110] softirqs last disabled at (1881303): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.122112] ---[ end trace 0000000000000000 ]--- <6> [298.122114] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.122183] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.122189] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.122236] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.122671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.122763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.122858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.122946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.123032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.123117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.123203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.123289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.123376] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.123484] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.123601] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.124608] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.135330] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.135574] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.136591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.136671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.136744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.136815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.136885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.136955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.137024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.137092] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.137160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.137230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.137301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.137370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.137446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.137521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.137595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.137670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.137744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.137818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.137891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.137973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.138054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.138134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.138216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.138295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.138377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.138466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.138545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.138625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.138705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.138782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.138861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.138939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.139022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.139106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.139190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.139271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.139351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.139432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.139515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.139594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.139676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.139755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.139834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.139912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.139996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.140077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.140156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.140236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.140318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.140394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.140478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.140557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.140635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.140712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.140788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.140865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.140940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.141016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.141095] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.141205] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.141209] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40936, lrc_seqno=40936, guc_id=0, flags=0x73 in no process [-1] <7> [298.141211] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.141271] ------------[ cut here ]------------ <4> [298.141272] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.141273] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.141346] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.141409] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.141421] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.141424] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.141425] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.141427] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.141432] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.141502] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.141503] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.141505] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.141507] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.141508] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.141509] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.141511] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.141512] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.141514] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.141515] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.141516] PKRU: 55555554 <4> [298.141518] Call Trace: <4> [298.141519] <TASK> <4> [298.141523] ? lock_sync+0x100/0x100 <4> [298.141527] ? lock_release+0xd0/0x2b0 <4> [298.141532] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.141538] process_one_work+0x239/0x760 <4> [298.141544] worker_thread+0x200/0x3f0 <4> [298.141547] ? __pfx_worker_thread+0x10/0x10 <4> [298.141549] kthread+0x10d/0x150 <4> [298.141552] ? __pfx_kthread+0x10/0x10 <4> [298.141556] ret_from_fork+0x3d4/0x480 <4> [298.141558] ? __pfx_kthread+0x10/0x10 <4> [298.141561] ret_from_fork_asm+0x1a/0x30 <4> [298.141569] </TASK> <4> [298.141570] irq event stamp: 1887167 <4> [298.141571] hardirqs last enabled at (1887173): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.141574] hardirqs last disabled at (1887178): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.141576] softirqs last enabled at (1886150): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.141579] softirqs last disabled at (1886143): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.141581] ---[ end trace 0000000000000000 ]--- <6> [298.141583] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.141652] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.141658] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.142900] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.143005] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.143111] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.143321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.143415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.143549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.143642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.143735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.143824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.143912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.144000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.144083] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.144180] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.144297] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.145309] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.155426] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.155685] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.156841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.156922] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.156990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.157060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.157131] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.157202] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.157273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.157343] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.157413] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.157506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.157593] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.157668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.157743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.157818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.157893] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.157971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.158047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.158121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.158195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.158277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.158358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.158445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.158535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.158614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.158693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.158772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.158853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.158934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.159015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.159095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.159175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.159253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.159338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.159429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.159521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.159602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.159682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.159765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.159848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.159929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.160010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.160089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.160168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.160243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.160324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.160401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.160499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.160595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.160674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.160750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.160831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.160913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.160993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.161073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.161150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.161229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.161305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.161382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.161480] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.161602] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.161606] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40936, lrc_seqno=40936, guc_id=0, flags=0x73 in no process [-1] <7> [298.161609] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.161668] ------------[ cut here ]------------ <4> [298.161669] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.161671] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:14/2466 <4> [298.161743] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.161809] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.161818] CPU: 6 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.161821] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.161822] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.161823] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.161829] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.161898] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.161900] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.161903] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.161904] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.161905] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.161907] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.161908] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.161909] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [298.161911] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.161912] CR2: 000077f1a4387048 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [298.161914] PKRU: 55555554 <4> [298.161915] Call Trace: <4> [298.161916] <TASK> <4> [298.161920] ? lock_sync+0x100/0x100 <4> [298.161925] ? lock_release+0xd0/0x2b0 <4> [298.161930] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.161936] process_one_work+0x239/0x760 <4> [298.161942] worker_thread+0x200/0x3f0 <4> [298.161945] ? __pfx_worker_thread+0x10/0x10 <4> [298.161947] kthread+0x10d/0x150 <4> [298.161950] ? __pfx_kthread+0x10/0x10 <4> [298.161954] ret_from_fork+0x3d4/0x480 <4> [298.161956] ? __pfx_kthread+0x10/0x10 <4> [298.161959] ret_from_fork_asm+0x1a/0x30 <4> [298.161966] </TASK> <4> [298.161968] irq event stamp: 1028429 <4> [298.161969] hardirqs last enabled at (1028435): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.161972] hardirqs last disabled at (1028440): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.161974] softirqs last enabled at (1027556): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.161976] softirqs last disabled at (1027551): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.161979] ---[ end trace 0000000000000000 ]--- <5> [298.162437] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40991, lrc_seqno=40991, guc_id=0, flags=0x73 in no process [-1] <7> [298.162445] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.162806] ------------[ cut here ]------------ <4> [298.162808] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.162812] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.163179] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.163255] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.163264] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.163267] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.163268] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.163270] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.163275] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <7> [298.163301] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [298.163382] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.163385] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.163388] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.163390] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.163392] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.163393] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.163395] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.163396] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.163398] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.163400] CR2: 000072fbb0ae0000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.163402] PKRU: 55555554 <4> [298.163404] Call Trace: <4> [298.163405] <TASK> <4> [298.163410] ? lock_sync+0x100/0x100 <4> [298.163425] ? lock_release+0xd0/0x2b0 <4> [298.163434] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.163444] process_one_work+0x239/0x760 <4> [298.163455] worker_thread+0x200/0x3f0 <4> [298.163463] ? __pfx_worker_thread+0x10/0x10 <4> [298.163489] kthread+0x10d/0x150 <4> [298.163493] ? __pfx_kthread+0x10/0x10 <4> [298.163499] ret_from_fork+0x3d4/0x480 <4> [298.163502] ? __pfx_kthread+0x10/0x10 <4> [298.163507] ret_from_fork_asm+0x1a/0x30 <4> [298.163518] </TASK> <4> [298.163520] irq event stamp: 1030331 <4> [298.163521] hardirqs last enabled at (1030337): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.163525] hardirqs last disabled at (1030342): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.163529] softirqs last enabled at (1027556): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.163532] softirqs last disabled at (1027551): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.163536] ---[ end trace 0000000000000000 ]--- <7> [298.163489] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [298.163539] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.163649] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.163657] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.163877] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.164082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.164172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.164267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.164355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.164472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.164558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.164645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.164732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.164814] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.164910] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.165029] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.166053] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.176779] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.177030] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.178011] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.178090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.178168] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.178246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.178347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.178478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.178570] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.178647] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.178724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.178798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.178875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.178953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.179030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.179105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.179181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.179256] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.179332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.179410] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.179504] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.179587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.179670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.179748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.179828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.179905] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.179983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.180063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.180144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.180225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.180307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.180387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.180479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.180559] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.180644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.180729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.180817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.180903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.180984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.181064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.181143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.181220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.181300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.181379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.181471] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.181552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.181635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.181714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.181792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.181868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.181947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.182024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.182099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.182178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.182258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.182338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.182429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.182510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.182586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.182661] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.182739] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.182853] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.182857] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40991, lrc_seqno=40991, guc_id=0, flags=0x73 in no process [-1] <7> [298.182860] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.182918] ------------[ cut here ]------------ <4> [298.182919] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.182921] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.182991] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.183056] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.183064] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.183067] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.183068] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.183070] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.183075] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.183143] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.183145] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.183147] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.183149] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.183150] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.183151] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.183152] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.183154] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.183155] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.183157] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.183158] PKRU: 55555554 <4> [298.183159] Call Trace: <4> [298.183160] <TASK> <4> [298.183164] ? lock_sync+0x100/0x100 <4> [298.183169] ? lock_release+0xd0/0x2b0 <4> [298.183174] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.183180] process_one_work+0x239/0x760 <4> [298.183186] worker_thread+0x200/0x3f0 <4> [298.183189] ? __pfx_worker_thread+0x10/0x10 <4> [298.183191] kthread+0x10d/0x150 <4> [298.183194] ? __pfx_kthread+0x10/0x10 <4> [298.183198] ret_from_fork+0x3d4/0x480 <4> [298.183200] ? __pfx_kthread+0x10/0x10 <4> [298.183203] ret_from_fork_asm+0x1a/0x30 <4> [298.183211] </TASK> <4> [298.183212] irq event stamp: 1033415 <4> [298.183213] hardirqs last enabled at (1033421): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.183216] hardirqs last disabled at (1033426): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.183218] softirqs last enabled at (1032336): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.183220] softirqs last disabled at (1032329): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.183222] ---[ end trace 0000000000000000 ]--- <6> [298.183224] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.183295] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.183301] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.183711] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.183915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.184007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.184101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.184191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.184279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.184365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.184460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.184550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.184637] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.184732] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.184847] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.185852] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.196574] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.196816] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.197962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.198041] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.198118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.198195] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.198272] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.198347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.198430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.198505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.198580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.198655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.198730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.198804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.198879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.198953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.199026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.199100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.199174] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.199248] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.199322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.199403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.199502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.199584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.199665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.199743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.199822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.199902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.199989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.200070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.200152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.200230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.200308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.200386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.200499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.200624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.200717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.200803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.200885] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.200964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.201043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.201120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.201200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.201279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.201357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.201439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.201520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.201597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.201675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.201751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.201833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.201911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.201988] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.202069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.202146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.202224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.202298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.202375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.202462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.202541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.202623] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.202735] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.202739] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=40991, lrc_seqno=40991, guc_id=0, flags=0x73 in no process [-1] <7> [298.202742] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.202800] ------------[ cut here ]------------ <4> [298.202801] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.202802] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.202874] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.202938] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.202947] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.202950] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.202951] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.202952] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.202957] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.203028] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.203030] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.203032] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.203033] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.203035] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.203036] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.203037] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.203038] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.203040] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.203041] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.203043] PKRU: 55555554 <4> [298.203044] Call Trace: <4> [298.203045] <TASK> <4> [298.203049] ? lock_sync+0x100/0x100 <4> [298.203054] ? lock_release+0xd0/0x2b0 <4> [298.203059] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.203065] process_one_work+0x239/0x760 <4> [298.203071] worker_thread+0x200/0x3f0 <4> [298.203073] ? __pfx_worker_thread+0x10/0x10 <4> [298.203076] kthread+0x10d/0x150 <4> [298.203079] ? __pfx_kthread+0x10/0x10 <4> [298.203082] ret_from_fork+0x3d4/0x480 <4> [298.203084] ? __pfx_kthread+0x10/0x10 <4> [298.203088] ret_from_fork_asm+0x1a/0x30 <4> [298.203095] </TASK> <4> [298.203096] irq event stamp: 1036525 <4> [298.203097] hardirqs last enabled at (1036531): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.203100] hardirqs last disabled at (1036536): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.203102] softirqs last enabled at (1035552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.203105] softirqs last disabled at (1035545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.203107] ---[ end trace 0000000000000000 ]--- <5> [298.204471] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41046, lrc_seqno=41046, guc_id=0, flags=0x73 in no process [-1] <7> [298.204474] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.204534] ------------[ cut here ]------------ <4> [298.204536] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.204537] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.204609] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.204672] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.204680] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.204682] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.204683] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.204685] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.204689] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.204758] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.204759] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.204761] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.204763] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.204764] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.204765] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.204766] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.204768] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.204769] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.204770] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.204772] PKRU: 55555554 <4> [298.204773] Call Trace: <4> [298.204774] <TASK> <4> [298.204778] ? lock_sync+0x100/0x100 <4> [298.204782] ? lock_release+0xd0/0x2b0 <4> [298.204787] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.204792] process_one_work+0x239/0x760 <4> [298.204798] worker_thread+0x200/0x3f0 <4> [298.204801] ? __pfx_worker_thread+0x10/0x10 <4> [298.204803] kthread+0x10d/0x150 <4> [298.204806] ? __pfx_kthread+0x10/0x10 <4> [298.204810] ret_from_fork+0x3d4/0x480 <4> [298.204812] ? __pfx_kthread+0x10/0x10 <4> [298.204815] ret_from_fork_asm+0x1a/0x30 <4> [298.204822] </TASK> <4> [298.204824] irq event stamp: 1038291 <4> [298.204825] hardirqs last enabled at (1038297): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.204827] hardirqs last disabled at (1038302): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.204829] softirqs last enabled at (1035552): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.204832] softirqs last disabled at (1035545): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.204834] ---[ end trace 0000000000000000 ]--- <6> [298.204836] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.204904] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.204910] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.204926] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.205029] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.205134] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.205342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.205439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.205535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.205624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.205713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.205801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.205891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.205981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.206064] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.206160] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.206277] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.207287] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.217423] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.217673] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.218605] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.218684] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.218761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.218837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.218914] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.218989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.219064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.219139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.219216] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.219293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.219368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.219451] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.219526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.219600] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.219674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.219749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.219822] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.219896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.219969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.220050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.220130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.220210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.220293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.220374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.220461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.220540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.220617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.220696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.220776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.220854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.220934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.221012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.221095] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.221179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.221262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.221342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.221424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.221503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.221586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.221665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.221747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.221826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.221904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.221980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.222060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.222136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.222213] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.222290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.222369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.222451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.222529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.222608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.222685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.222766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.222845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.222923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.223000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.223076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.223155] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.223267] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.223271] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41046, lrc_seqno=41046, guc_id=0, flags=0x73 in no process [-1] <7> [298.223273] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.223332] ------------[ cut here ]------------ <4> [298.223333] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.223334] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.223406] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.223474] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.223482] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.223485] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.223486] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.223488] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.223493] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.223563] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.223565] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.223567] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.223569] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.223570] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.223571] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.223573] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.223574] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.223576] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.223577] CR2: 000072fbb0ae1000 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [298.223579] PKRU: 55555554 <4> [298.223580] Call Trace: <4> [298.223581] <TASK> <4> [298.223585] ? lock_sync+0x100/0x100 <4> [298.223589] ? lock_release+0xd0/0x2b0 <4> [298.223595] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.223601] process_one_work+0x239/0x760 <4> [298.223607] worker_thread+0x200/0x3f0 <4> [298.223610] ? __pfx_worker_thread+0x10/0x10 <4> [298.223612] kthread+0x10d/0x150 <4> [298.223615] ? __pfx_kthread+0x10/0x10 <4> [298.223619] ret_from_fork+0x3d4/0x480 <4> [298.223621] ? __pfx_kthread+0x10/0x10 <4> [298.223624] ret_from_fork_asm+0x1a/0x30 <4> [298.223632] </TASK> <4> [298.223633] irq event stamp: 839731 <4> [298.223634] hardirqs last enabled at (839737): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.223637] hardirqs last disabled at (839742): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.223639] softirqs last enabled at (838610): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.223641] softirqs last disabled at (838603): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.223644] ---[ end trace 0000000000000000 ]--- <6> [298.223645] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.223715] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.223722] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.224955] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.225060] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.225165] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.225373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.225480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.225576] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.225666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.225757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.225848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.225937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.226026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.226114] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.226211] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.226329] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.227330] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.237425] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.237673] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.238861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.238940] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.239010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.239082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.239151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.239220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.239288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.239356] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.239430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.239505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.239583] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.239661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.239737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.239813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.239888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.239962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.240037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.240111] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.240185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.240267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.240349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.240438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.240523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.240603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.240682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.240761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.240839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.240919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.241000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.241078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.241158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.241237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.241324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.241412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.241519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.241603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.241684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.241763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.241840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.241917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.241997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.242076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.242152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.242230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.242310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.242387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.242477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.242560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.242643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.242721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.242799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.242884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.242967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.243049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.243127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.243208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.243287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.243363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.243451] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.243559] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.243564] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41046, lrc_seqno=41046, guc_id=0, flags=0x73 in no process [-1] <7> [298.243566] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.243626] ------------[ cut here ]------------ <4> [298.243627] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.243629] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.243696] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.243755] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.243763] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.243765] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.243766] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.243768] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.243773] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.243836] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.243838] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.243840] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.243841] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.243842] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.243843] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.243844] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.243846] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.243847] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.243848] CR2: 000072fbb0ae0000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.243850] PKRU: 55555554 <4> [298.243851] Call Trace: <4> [298.243852] <TASK> <4> [298.243856] ? lock_sync+0x100/0x100 <4> [298.243860] ? lock_release+0xd0/0x2b0 <4> [298.243865] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.243870] process_one_work+0x239/0x760 <4> [298.243876] worker_thread+0x200/0x3f0 <4> [298.243878] ? __pfx_worker_thread+0x10/0x10 <4> [298.243881] kthread+0x10d/0x150 <4> [298.243883] ? __pfx_kthread+0x10/0x10 <4> [298.243887] ret_from_fork+0x3d4/0x480 <4> [298.243888] ? __pfx_kthread+0x10/0x10 <4> [298.243891] ret_from_fork_asm+0x1a/0x30 <4> [298.243898] </TASK> <4> [298.243899] irq event stamp: 1041765 <4> [298.243900] hardirqs last enabled at (1041771): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.243903] hardirqs last disabled at (1041776): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.243905] softirqs last enabled at (1040814): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.243907] softirqs last disabled at (1040807): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.243909] ---[ end trace 0000000000000000 ]--- <5> [298.244290] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41101, lrc_seqno=41101, guc_id=0, flags=0x73 in no process [-1] <7> [298.244325] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.244647] ------------[ cut here ]------------ <4> [298.244650] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.244652] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.244743] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.244815] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.244825] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.244828] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.244829] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.244831] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.244839] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.244917] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.244919] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.244923] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.244925] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.244926] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.244927] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.244929] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.244931] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.244933] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.244934] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.244936] PKRU: 55555554 <4> [298.244937] Call Trace: <4> [298.244939] <TASK> <4> [298.244944] ? lock_sync+0x100/0x100 <4> [298.244951] ? lock_release+0xd0/0x2b0 <4> [298.244959] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.244965] process_one_work+0x239/0x760 <4> [298.244972] worker_thread+0x200/0x3f0 <4> [298.244975] ? __pfx_worker_thread+0x10/0x10 <4> [298.244978] kthread+0x10d/0x150 <4> [298.244981] ? __pfx_kthread+0x10/0x10 <4> [298.244984] ret_from_fork+0x3d4/0x480 <4> [298.244986] ? __pfx_kthread+0x10/0x10 <4> [298.244990] ret_from_fork_asm+0x1a/0x30 <4> [298.244998] </TASK> <4> [298.245001] irq event stamp: 1889227 <4> [298.245002] hardirqs last enabled at (1889233): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.245006] hardirqs last disabled at (1889238): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.245008] softirqs last enabled at (1888436): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.245011] softirqs last disabled at (1888431): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.245013] ---[ end trace 0000000000000000 ]--- <6> [298.245017] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.245114] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.245125] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.245389] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.245536] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.245691] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.245915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.246014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.246113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.246206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.246299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.246388] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.246493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.246586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.246674] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.246774] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.246897] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.247925] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.258654] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.258908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.259885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.259970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.260050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.260133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.260214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.260295] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.260376] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.260469] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.260551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.260632] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.260712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.260791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.260871] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.260951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.261029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.261108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.261185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.261263] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.261340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.261429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.261515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.261602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.261690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.261775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.261857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.261940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.262022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.262106] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.262192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.262275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.262362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.262452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.262542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.262631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.262722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.262812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.262898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.262981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.263063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.263144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.263229] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.263312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.263399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.263495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.263582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.263665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.263748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.263829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.263912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.263993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.264074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.264163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.264247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.264330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.264411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.264507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.264589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.264669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.264753] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.264870] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.264875] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41101, lrc_seqno=41101, guc_id=0, flags=0x73 in no process [-1] <7> [298.264877] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.264938] ------------[ cut here ]------------ <4> [298.264940] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.264941] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.265016] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.265082] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.265090] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.265093] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.265094] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.265096] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.265101] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.265176] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.265178] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.265180] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.265181] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.265183] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.265184] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.265185] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.265187] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.265188] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.265190] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.265191] PKRU: 55555554 <4> [298.265192] Call Trace: <4> [298.265193] <TASK> <4> [298.265198] ? lock_sync+0x100/0x100 <4> [298.265202] ? lock_release+0xd0/0x2b0 <4> [298.265208] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.265214] process_one_work+0x239/0x760 <4> [298.265220] worker_thread+0x200/0x3f0 <4> [298.265223] ? __pfx_worker_thread+0x10/0x10 <4> [298.265225] kthread+0x10d/0x150 <4> [298.265228] ? __pfx_kthread+0x10/0x10 <4> [298.265232] ret_from_fork+0x3d4/0x480 <4> [298.265234] ? __pfx_kthread+0x10/0x10 <4> [298.265237] ret_from_fork_asm+0x1a/0x30 <4> [298.265245] </TASK> <4> [298.265246] irq event stamp: 1892323 <4> [298.265247] hardirqs last enabled at (1892329): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.265250] hardirqs last disabled at (1892334): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.265252] softirqs last enabled at (1891432): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.265255] softirqs last disabled at (1891425): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.265257] ---[ end trace 0000000000000000 ]--- <6> [298.265259] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.265334] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.265340] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.266580] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.266694] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.266806] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.267013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.267102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.267219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.267344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.267445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.267534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.267623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.267713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.267797] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.267893] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.268012] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.269024] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.279753] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.279995] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.281175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.281247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.281316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.281387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.281473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.281553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.281631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.281706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.281781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.281856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.281931] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.282005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.282082] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.282161] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.282237] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.282312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.282387] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.282474] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.282549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.282631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.282712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.282793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.282873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.282951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.283029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.283110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.283191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.283272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.283352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.283436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.283515] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.283594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.283678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.283762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.283847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.283928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.284008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.284087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.284168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.284249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.284332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.284419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.284501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.284580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.284662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.284740] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.284819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.284897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.284976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.285057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.285139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.285223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.285304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.285387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.285477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.285557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.285635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.285712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.285793] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.285904] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.285909] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41101, lrc_seqno=41101, guc_id=0, flags=0x73 in no process [-1] <7> [298.285911] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.285969] ------------[ cut here ]------------ <4> [298.285970] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.285971] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.286042] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.286106] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.286114] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.286117] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.286118] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.286120] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.286125] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.286194] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.286196] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.286198] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.286199] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.286201] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.286202] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.286203] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.286205] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.286206] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.286208] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.286209] PKRU: 55555554 <4> [298.286210] Call Trace: <4> [298.286211] <TASK> <4> [298.286215] ? lock_sync+0x100/0x100 <4> [298.286220] ? lock_release+0xd0/0x2b0 <4> [298.286225] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.286231] process_one_work+0x239/0x760 <4> [298.286237] worker_thread+0x200/0x3f0 <4> [298.286240] ? __pfx_worker_thread+0x10/0x10 <4> [298.286242] kthread+0x10d/0x150 <4> [298.286245] ? __pfx_kthread+0x10/0x10 <4> [298.286249] ret_from_fork+0x3d4/0x480 <4> [298.286251] ? __pfx_kthread+0x10/0x10 <4> [298.286254] ret_from_fork_asm+0x1a/0x30 <4> [298.286262] </TASK> <4> [298.286263] irq event stamp: 1046117 <4> [298.286264] hardirqs last enabled at (1046123): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.286267] hardirqs last disabled at (1046128): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.286269] softirqs last enabled at (1045220): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.286272] softirqs last disabled at (1045199): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.286274] ---[ end trace 0000000000000000 ]--- <5> [298.287638] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41156, lrc_seqno=41156, guc_id=0, flags=0x73 in no process [-1] <7> [298.287641] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.287701] ------------[ cut here ]------------ <4> [298.287702] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.287703] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.287773] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.287837] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.287845] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.287847] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.287848] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.287849] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.287854] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.287922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.287924] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.287926] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.287927] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.287928] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.287929] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.287931] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.287932] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.287933] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.287935] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.287936] PKRU: 55555554 <4> [298.287937] Call Trace: <4> [298.287938] <TASK> <4> [298.287942] ? lock_sync+0x100/0x100 <4> [298.287946] ? lock_release+0xd0/0x2b0 <4> [298.287951] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.287957] process_one_work+0x239/0x760 <4> [298.287963] worker_thread+0x200/0x3f0 <4> [298.287966] ? __pfx_worker_thread+0x10/0x10 <4> [298.287968] kthread+0x10d/0x150 <4> [298.287971] ? __pfx_kthread+0x10/0x10 <4> [298.287974] ret_from_fork+0x3d4/0x480 <4> [298.287976] ? __pfx_kthread+0x10/0x10 <4> [298.287980] ret_from_fork_asm+0x1a/0x30 <4> [298.287987] </TASK> <4> [298.287988] irq event stamp: 1048061 <4> [298.287989] hardirqs last enabled at (1048067): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.287992] hardirqs last disabled at (1048072): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.287994] softirqs last enabled at (1046412): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.287996] softirqs last disabled at (1046401): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.287999] ---[ end trace 0000000000000000 ]--- <6> [298.288000] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.288069] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.288075] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.288304] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.288408] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.288528] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.288731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.288822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.288914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.289003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.289091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.289212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.289307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.289399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.289504] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.289602] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.289720] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.290744] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.301425] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.301676] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.302697] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.302776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.302846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.302918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.302989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.303060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.303129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.303199] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.303267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.303335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.303405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.303501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.303579] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.303655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.303732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.303807] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.303881] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.303956] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.304030] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.304113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.304196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.304275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.304360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.304451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.304531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.304610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.304687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.304767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.304847] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.304925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.305005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.305083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.305169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.305253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.305337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.305437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.305538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.305620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.305700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.305780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.305860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.305938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.306017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.306099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.306185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.306264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.306343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.306433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.306512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.306589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.306664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.306749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.306826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.306899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.306970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.307045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.307118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.307189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.307263] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.307369] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.307373] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41156, lrc_seqno=41156, guc_id=0, flags=0x73 in no process [-1] <7> [298.307376] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.308288] ------------[ cut here ]------------ <4> [298.308289] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.308290] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.308357] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.308431] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.308440] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.308443] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.308444] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.308445] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.308451] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.308521] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.308523] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.308525] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.308526] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.308527] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.308529] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.308530] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.308531] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.308533] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.308534] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.308536] PKRU: 55555554 <4> [298.308537] Call Trace: <4> [298.308538] <TASK> <4> [298.308542] ? lock_sync+0x100/0x100 <4> [298.308547] ? lock_release+0xd0/0x2b0 <4> [298.308552] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.308559] process_one_work+0x239/0x760 <4> [298.308565] worker_thread+0x200/0x3f0 <4> [298.308567] ? __pfx_worker_thread+0x10/0x10 <4> [298.308569] kthread+0x10d/0x150 <4> [298.308572] ? __pfx_kthread+0x10/0x10 <4> [298.308575] ret_from_fork+0x3d4/0x480 <4> [298.308577] ? __pfx_kthread+0x10/0x10 <4> [298.308580] ret_from_fork_asm+0x1a/0x30 <4> [298.308587] </TASK> <4> [298.308588] irq event stamp: 1051225 <4> [298.308589] hardirqs last enabled at (1051231): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.308592] hardirqs last disabled at (1051236): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.308594] softirqs last enabled at (1050910): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.308596] softirqs last disabled at (1050905): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.308598] ---[ end trace 0000000000000000 ]--- <7> [298.308611] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.308706] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [298.308797] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.308864] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.308870] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.309276] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.309478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.309566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.309650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.309730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.309810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.309888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.309968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.310051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.310130] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.310218] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.310332] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.311355] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.321425] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.321672] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.322806] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.322886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.322961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.323033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.323103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.323171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.323240] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.323309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.323380] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.323466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.323543] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.323619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.323695] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.323767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.323842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.323917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.323992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.324066] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.324140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.324226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.324308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.324387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.324477] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.324555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.324632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.324708] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.324784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.324863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.324944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.325022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.325101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.325179] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.325263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.325348] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.325439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.325524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.325605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.325683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.325762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.325839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.325919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.325998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.326075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.326152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.326234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.326313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.326392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.326483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.326561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.326637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.326713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.326792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.326868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.326945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.327021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.327103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.327180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.327257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.327338] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.327457] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.327462] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41156, lrc_seqno=41156, guc_id=0, flags=0x73 in no process [-1] <7> [298.327464] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.327524] ------------[ cut here ]------------ <4> [298.327525] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.327526] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.327597] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.327661] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.327669] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.327672] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.327673] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.327675] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.327680] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.327748] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.327750] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.327752] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.327754] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.327755] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.327756] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.327757] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.327759] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.327760] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.327762] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.327763] PKRU: 55555554 <4> [298.327764] Call Trace: <4> [298.327765] <TASK> <4> [298.327769] ? lock_sync+0x100/0x100 <4> [298.327774] ? lock_release+0xd0/0x2b0 <4> [298.327779] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.327785] process_one_work+0x239/0x760 <4> [298.327791] worker_thread+0x200/0x3f0 <4> [298.327794] ? __pfx_worker_thread+0x10/0x10 <4> [298.327796] kthread+0x10d/0x150 <4> [298.327799] ? __pfx_kthread+0x10/0x10 <4> [298.327803] ret_from_fork+0x3d4/0x480 <4> [298.327805] ? __pfx_kthread+0x10/0x10 <4> [298.327808] ret_from_fork_asm+0x1a/0x30 <4> [298.327815] </TASK> <4> [298.327817] irq event stamp: 1054371 <4> [298.327818] hardirqs last enabled at (1054377): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.327820] hardirqs last disabled at (1054382): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.327823] softirqs last enabled at (1053532): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.327825] softirqs last disabled at (1053521): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.327827] ---[ end trace 0000000000000000 ]--- <5> [298.329047] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41211, lrc_seqno=41211, guc_id=0, flags=0x73 in no process [-1] <7> [298.329050] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.329110] ------------[ cut here ]------------ <4> [298.329111] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.329112] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.329182] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.329246] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.329254] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.329256] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.329257] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.329259] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.329263] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.329331] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.329333] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.329335] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.329336] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.329337] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.329338] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.329340] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.329341] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.329342] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.329344] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.329345] PKRU: 55555554 <4> [298.329346] Call Trace: <4> [298.329347] <TASK> <4> [298.329351] ? lock_sync+0x100/0x100 <4> [298.329355] ? lock_release+0xd0/0x2b0 <4> [298.329360] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.329366] process_one_work+0x239/0x760 <4> [298.329372] worker_thread+0x200/0x3f0 <4> [298.329374] ? __pfx_worker_thread+0x10/0x10 <4> [298.329377] kthread+0x10d/0x150 <4> [298.329380] ? __pfx_kthread+0x10/0x10 <4> [298.329383] ret_from_fork+0x3d4/0x480 <4> [298.329385] ? __pfx_kthread+0x10/0x10 <4> [298.329388] ret_from_fork_asm+0x1a/0x30 <4> [298.329396] </TASK> <4> [298.329397] irq event stamp: 1056307 <4> [298.329398] hardirqs last enabled at (1056313): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.329401] hardirqs last disabled at (1056318): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.329403] softirqs last enabled at (1053532): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.329405] softirqs last disabled at (1053521): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.329407] ---[ end trace 0000000000000000 ]--- <6> [298.329409] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <7> [298.329508] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.329614] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [298.329713] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.329719] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.329769] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.329992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.330090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.330190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.330286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.330380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.330480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.330573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.330671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.330762] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.330864] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.330987] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.331997] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.342423] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.342677] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.343649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.343736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.343819] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.343901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.343982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.344061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.344140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.344219] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.344298] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.344377] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.344471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.344550] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.344630] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.344712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.344793] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.344877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.344957] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.345036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.345114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.345200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.345286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.345370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.345465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.345552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.345635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.345717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.345798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.345883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.345968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.346052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.346135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.346221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.346315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.346407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.346510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.346597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.346684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.346771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.346856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.346940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.347026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.347110] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.347193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.347275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.347361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.347450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.347538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.347623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.347711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.347793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.347875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.347960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.348044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.348126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.348208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.348295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.348380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.348472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.348559] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.348674] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.348678] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41211, lrc_seqno=41211, guc_id=0, flags=0x73 in no process [-1] <7> [298.348680] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.348742] ------------[ cut here ]------------ <4> [298.348743] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.348745] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.348822] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.348886] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.348895] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.348898] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.348899] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.348901] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.348906] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.348981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.348983] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.348985] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.348987] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.348988] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.348989] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.348991] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.348992] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.348994] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.348995] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.348997] PKRU: 55555554 <4> [298.348998] Call Trace: <4> [298.348999] <TASK> <4> [298.349003] ? lock_sync+0x100/0x100 <4> [298.349008] ? lock_release+0xd0/0x2b0 <4> [298.349013] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.349019] process_one_work+0x239/0x760 <4> [298.349026] worker_thread+0x200/0x3f0 <4> [298.349028] ? __pfx_worker_thread+0x10/0x10 <4> [298.349031] kthread+0x10d/0x150 <4> [298.349034] ? __pfx_kthread+0x10/0x10 <4> [298.349037] ret_from_fork+0x3d4/0x480 <4> [298.349039] ? __pfx_kthread+0x10/0x10 <4> [298.349043] ret_from_fork_asm+0x1a/0x30 <4> [298.349050] </TASK> <4> [298.349051] irq event stamp: 845399 <4> [298.349052] hardirqs last enabled at (845405): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.349055] hardirqs last disabled at (845410): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.349058] softirqs last enabled at (844266): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.349060] softirqs last disabled at (844259): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.349062] ---[ end trace 0000000000000000 ]--- <6> [298.349064] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.349139] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.349145] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.350433] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.350544] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.350656] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.350862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.350955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.351053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.351147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.351241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.351334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.351431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.351528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.351620] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.351721] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.351844] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.352854] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.363420] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.363672] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.364797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.364879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.364949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.365019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.365090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.365162] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.365231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.365300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.365368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.365443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.365518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.365592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.365666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.365741] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.365814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.365888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.365962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.366040] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.366117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.366199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.366280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.366360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.366448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.366526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.366604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.366682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.366760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.366839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.366920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.367003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.367086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.367165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.367251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.367335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.367422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.367503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.367583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.367660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.367739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.367815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.367895] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.367972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.368048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.368124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.368204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.368281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.368359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.368444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.368524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.368602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.368680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.368763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.368843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.368921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.368998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.369075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.369151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.369227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.369306] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.369419] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.369423] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41211, lrc_seqno=41211, guc_id=0, flags=0x73 in no process [-1] <7> [298.369426] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.369484] ------------[ cut here ]------------ <4> [298.369485] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.369487] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.369558] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.369621] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.369630] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.369632] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.369633] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.369635] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.369640] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.369709] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.369711] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.369713] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.369715] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.369716] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.369717] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.369718] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.369720] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.369721] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.369723] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.369724] PKRU: 55555554 <4> [298.369725] Call Trace: <4> [298.369726] <TASK> <4> [298.369730] ? lock_sync+0x100/0x100 <4> [298.369735] ? lock_release+0xd0/0x2b0 <4> [298.369740] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.369746] process_one_work+0x239/0x760 <4> [298.369752] worker_thread+0x200/0x3f0 <4> [298.369755] ? __pfx_worker_thread+0x10/0x10 <4> [298.369757] kthread+0x10d/0x150 <4> [298.369760] ? __pfx_kthread+0x10/0x10 <4> [298.369764] ret_from_fork+0x3d4/0x480 <4> [298.369766] ? __pfx_kthread+0x10/0x10 <4> [298.369769] ret_from_fork_asm+0x1a/0x30 <4> [298.369776] </TASK> <4> [298.369777] irq event stamp: 848479 <4> [298.369778] hardirqs last enabled at (848485): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.369781] hardirqs last disabled at (848490): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.369783] softirqs last enabled at (847534): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.369786] softirqs last disabled at (847527): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.369788] ---[ end trace 0000000000000000 ]--- <5> [298.371143] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41266, lrc_seqno=41266, guc_id=0, flags=0x73 in no process [-1] <7> [298.371145] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.371205] ------------[ cut here ]------------ <4> [298.371206] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.371208] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.371280] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.371343] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.371350] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.371353] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.371354] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.371355] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.371360] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.371437] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.371439] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.371441] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.371443] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.371444] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.371445] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.371447] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.371448] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.371449] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.371451] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.371452] PKRU: 55555554 <4> [298.371453] Call Trace: <4> [298.371454] <TASK> <4> [298.371458] ? lock_sync+0x100/0x100 <4> [298.371462] ? lock_release+0xd0/0x2b0 <4> [298.371468] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.371473] process_one_work+0x239/0x760 <4> [298.371479] worker_thread+0x200/0x3f0 <4> [298.371482] ? __pfx_worker_thread+0x10/0x10 <4> [298.371484] kthread+0x10d/0x150 <4> [298.371487] ? __pfx_kthread+0x10/0x10 <4> [298.371491] ret_from_fork+0x3d4/0x480 <4> [298.371493] ? __pfx_kthread+0x10/0x10 <4> [298.371497] ret_from_fork_asm+0x1a/0x30 <4> [298.371504] </TASK> <4> [298.371505] irq event stamp: 850401 <4> [298.371506] hardirqs last enabled at (850407): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.371509] hardirqs last disabled at (850412): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.371511] softirqs last enabled at (850240): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.371513] softirqs last disabled at (850233): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.371516] ---[ end trace 0000000000000000 ]--- <6> [298.371517] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.371588] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.371594] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.371829] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.371933] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.372040] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.372244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.372332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.372428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.372517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.372605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.372693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.372781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.372871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.372957] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.373054] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.373169] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.374175] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.384419] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.384668] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.385691] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.385769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.385847] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.385925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.386002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.386078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.386155] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.386231] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.386309] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.386386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.386473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.386549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.386625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.386700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.386773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.386849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.386926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.387001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.387078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.387163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.387244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.387323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.387404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.387494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.387572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.387651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.387731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.387812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.387893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.387972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.388051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.388129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.388215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.388300] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.388387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.388479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.388562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.388641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.388719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.388795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.388875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.388953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.389031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.389107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.389187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.389264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.389340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.389418] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.389501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.389580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.389658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.389738] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.389817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.389893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.389969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.390047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.390123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.390202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.390284] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.390395] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.390399] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41266, lrc_seqno=41266, guc_id=0, flags=0x73 in no process [-1] <7> [298.390402] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.391331] ------------[ cut here ]------------ <4> [298.391332] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.391334] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.391408] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.391477] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.391486] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.391488] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.391489] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.391491] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.391496] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.391567] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.391569] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.391571] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.391573] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.391574] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.391575] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.391576] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.391578] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.391579] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.391580] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.391582] PKRU: 55555554 <4> [298.391583] Call Trace: <4> [298.391584] <TASK> <4> [298.391588] ? lock_sync+0x100/0x100 <4> [298.391592] ? lock_release+0xd0/0x2b0 <4> [298.391598] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.391603] process_one_work+0x239/0x760 <4> [298.391609] worker_thread+0x200/0x3f0 <4> [298.391612] ? __pfx_worker_thread+0x10/0x10 <4> [298.391615] kthread+0x10d/0x150 <4> [298.391618] ? __pfx_kthread+0x10/0x10 <4> [298.391621] ret_from_fork+0x3d4/0x480 <4> [298.391623] ? __pfx_kthread+0x10/0x10 <4> [298.391627] ret_from_fork_asm+0x1a/0x30 <4> [298.391634] </TASK> <4> [298.391635] irq event stamp: 853493 <4> [298.391636] hardirqs last enabled at (853499): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.391639] hardirqs last disabled at (853504): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.391641] softirqs last enabled at (852738): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.391644] softirqs last disabled at (852731): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.391646] ---[ end trace 0000000000000000 ]--- <6> [298.391647] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.391718] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.391724] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.391737] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.391838] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.392130] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.392345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.392450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.392552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.392650] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.392746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.392837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.392928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.393021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.393108] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.393208] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.393328] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.394335] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.404418] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.404671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.405746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.405831] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.405911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.405988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.406064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.406140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.406213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.406286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.406359] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.406437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.406517] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.406597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.406675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.406754] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.406833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.406912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.406991] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.407073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.407154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.407243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.407332] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.407419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.407505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.407588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.407670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.407756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.407842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.407929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.408016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.408100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.408185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.408269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.408358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.408454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.408543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.408630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.408714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.408796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.408879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.408960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.409044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.409129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.409215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.409299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.409386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.409479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.409563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.409645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.409729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.409815] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.409899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.409985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.410069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.410152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.410233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.410316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.410399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.410491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.410576] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.410693] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.410697] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41266, lrc_seqno=41266, guc_id=0, flags=0x73 in no process [-1] <7> [298.410699] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.410761] ------------[ cut here ]------------ <4> [298.410762] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.410763] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.410840] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.410904] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.410912] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.410915] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.410916] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.410918] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.410923] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.410997] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.410998] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.411001] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.411002] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.411003] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.411004] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.411006] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.411007] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.411009] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.411010] CR2: 00007f74cf18b008 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.411012] PKRU: 55555554 <4> [298.411013] Call Trace: <4> [298.411014] <TASK> <4> [298.411018] ? lock_sync+0x100/0x100 <4> [298.411022] ? lock_release+0xd0/0x2b0 <4> [298.411028] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.411034] process_one_work+0x239/0x760 <4> [298.411040] worker_thread+0x200/0x3f0 <4> [298.411043] ? __pfx_worker_thread+0x10/0x10 <4> [298.411045] kthread+0x10d/0x150 <4> [298.411048] ? __pfx_kthread+0x10/0x10 <4> [298.411052] ret_from_fork+0x3d4/0x480 <4> [298.411054] ? __pfx_kthread+0x10/0x10 <4> [298.411058] ret_from_fork_asm+0x1a/0x30 <4> [298.411065] </TASK> <4> [298.411066] irq event stamp: 1898613 <4> [298.411067] hardirqs last enabled at (1898619): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.411070] hardirqs last disabled at (1898624): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.411073] softirqs last enabled at (1897734): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.411075] softirqs last disabled at (1897727): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.411077] ---[ end trace 0000000000000000 ]--- <5> [298.412436] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41321, lrc_seqno=41321, guc_id=0, flags=0x73 in no process [-1] <7> [298.412439] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.412504] ------------[ cut here ]------------ <4> [298.412505] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.412507] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.412580] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.412642] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.412650] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.412653] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.412654] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.412655] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.412660] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.412732] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.412734] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.412736] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.412737] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.412739] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.412740] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.412741] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.412742] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.412744] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.412745] CR2: 00007f74cf18b008 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.412746] PKRU: 55555554 <4> [298.412748] Call Trace: <4> [298.412749] <TASK> <4> [298.412752] ? lock_sync+0x100/0x100 <4> [298.412757] ? lock_release+0xd0/0x2b0 <4> [298.412762] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.412768] process_one_work+0x239/0x760 <4> [298.412774] worker_thread+0x200/0x3f0 <4> [298.412776] ? __pfx_worker_thread+0x10/0x10 <4> [298.412779] kthread+0x10d/0x150 <4> [298.412782] ? __pfx_kthread+0x10/0x10 <4> [298.412785] ret_from_fork+0x3d4/0x480 <4> [298.412787] ? __pfx_kthread+0x10/0x10 <4> [298.412791] ret_from_fork_asm+0x1a/0x30 <4> [298.412798] </TASK> <4> [298.412799] irq event stamp: 1900527 <4> [298.412800] hardirqs last enabled at (1900533): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.412803] hardirqs last disabled at (1900538): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.412805] softirqs last enabled at (1897734): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.412808] softirqs last disabled at (1897727): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.412810] ---[ end trace 0000000000000000 ]--- <6> [298.412811] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.412884] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.412890] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.413088] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.413196] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.413307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.413518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.413615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.413714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.413808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.413901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.413991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.414083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.414175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.414262] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.414363] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.414498] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.415509] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.425418] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.425668] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.426671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.426751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.426829] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.426907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.426984] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.427061] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.427136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.427214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.427292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.427370] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.427456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.427532] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.427610] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.427688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.427765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.427842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.427916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.427990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.428064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.428147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.428230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.428311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.428393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.428483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.428562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.428640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.428721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.428804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.428888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.428967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.429046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.429125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.429212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.429297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.429381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.429470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.429550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.429627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.429704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.429780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.429860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.429942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.430025] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.430104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.430186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.430263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.430340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.430423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.430502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.430580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.430659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.430742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.430821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.430900] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.430981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.431062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.431138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.431215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.431295] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.431405] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.432624] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41321, lrc_seqno=41321, guc_id=0, flags=0x73 in no process [-1] <7> [298.432627] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.432687] ------------[ cut here ]------------ <4> [298.432688] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.432689] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.432761] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.432825] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.432833] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.432836] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.432838] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.432839] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.432844] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.432914] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.432916] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.432918] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.432919] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.432921] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.432922] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.432923] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.432924] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.432926] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.432927] CR2: 00007f74cf18b008 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.432929] PKRU: 55555554 <4> [298.432930] Call Trace: <4> [298.432931] <TASK> <4> [298.432935] ? lock_sync+0x100/0x100 <4> [298.432939] ? lock_release+0xd0/0x2b0 <4> [298.432944] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.432950] process_one_work+0x239/0x760 <4> [298.432956] worker_thread+0x200/0x3f0 <4> [298.432959] ? __pfx_worker_thread+0x10/0x10 <4> [298.432961] kthread+0x10d/0x150 <4> [298.432964] ? __pfx_kthread+0x10/0x10 <4> [298.432968] ret_from_fork+0x3d4/0x480 <4> [298.432970] ? __pfx_kthread+0x10/0x10 <4> [298.432973] ret_from_fork_asm+0x1a/0x30 <4> [298.432981] </TASK> <4> [298.432982] irq event stamp: 1903663 <4> [298.432983] hardirqs last enabled at (1903669): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.432986] hardirqs last disabled at (1903674): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.432988] softirqs last enabled at (1902858): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.432990] softirqs last disabled at (1902853): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.432993] ---[ end trace 0000000000000000 ]--- <6> [298.432994] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.433065] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.433072] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.433086] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.433189] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.433616] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.433826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.433920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.434014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.434102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.434188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.434274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.434360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.434472] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.434565] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.434661] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.434780] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.435792] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.446419] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.446666] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.447815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.447894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.447964] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.448037] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.448108] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.448178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.448246] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.448315] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.448383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.448468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.448555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.448634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.448711] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.448786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.448860] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.448935] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.449009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.449083] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.449157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.449239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.449321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.449403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.449505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.449595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.449673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.449754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.449834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.449917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.449998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.450078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.450157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.450237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.450322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.450406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.450512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.450608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.450693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.450773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.450851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.450928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.451009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.451088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.451166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.451243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.451327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.451414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.451506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.451584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.451663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.451741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.451817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.451896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.451973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.452050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.452126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.452204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.452280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.452356] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.452450] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.452570] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.452574] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41321, lrc_seqno=41321, guc_id=0, flags=0x73 in no process [-1] <7> [298.452577] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.452635] ------------[ cut here ]------------ <4> [298.452636] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.452638] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.452710] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.452776] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.452784] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.452787] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.452788] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.452789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.452795] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.452865] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.452866] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.452869] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.452870] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.452871] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.452873] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.452874] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.452875] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.452877] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.452878] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.452880] PKRU: 55555554 <4> [298.452881] Call Trace: <4> [298.452882] <TASK> <4> [298.452886] ? lock_sync+0x100/0x100 <4> [298.452891] ? lock_release+0xd0/0x2b0 <4> [298.452896] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.452902] process_one_work+0x239/0x760 <4> [298.452908] worker_thread+0x200/0x3f0 <4> [298.452911] ? __pfx_worker_thread+0x10/0x10 <4> [298.452914] kthread+0x10d/0x150 <4> [298.452916] ? __pfx_kthread+0x10/0x10 <4> [298.452920] ret_from_fork+0x3d4/0x480 <4> [298.452922] ? __pfx_kthread+0x10/0x10 <4> [298.452925] ret_from_fork_asm+0x1a/0x30 <4> [298.452933] </TASK> <4> [298.452934] irq event stamp: 1061917 <4> [298.452935] hardirqs last enabled at (1061923): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.452938] hardirqs last disabled at (1061928): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.452940] softirqs last enabled at (1061044): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.452943] softirqs last disabled at (1061023): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.452945] ---[ end trace 0000000000000000 ]--- <5> [298.454189] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41376, lrc_seqno=41376, guc_id=0, flags=0x73 in no process [-1] <7> [298.454192] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.454252] ------------[ cut here ]------------ <4> [298.454254] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.454255] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.454325] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.454388] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.454396] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.454398] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.454400] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.454401] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.454406] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.454506] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.454509] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.454513] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.454516] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.454518] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.454520] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.454522] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.454525] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.454527] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.454530] CR2: 000072fbb0ae0000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [298.454532] PKRU: 55555554 <4> [298.454535] Call Trace: <4> [298.454537] <TASK> <4> [298.454543] ? lock_sync+0x100/0x100 <4> [298.454549] ? lock_release+0xd0/0x2b0 <4> [298.454555] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.454565] process_one_work+0x239/0x760 <4> [298.454576] worker_thread+0x200/0x3f0 <4> [298.454581] ? __pfx_worker_thread+0x10/0x10 <4> [298.454585] kthread+0x10d/0x150 <4> [298.454590] ? __pfx_kthread+0x10/0x10 <4> [298.454596] ret_from_fork+0x3d4/0x480 <4> [298.454599] ? __pfx_kthread+0x10/0x10 <4> [298.454604] ret_from_fork_asm+0x1a/0x30 <4> [298.454616] </TASK> <4> [298.454617] irq event stamp: 1063847 <4> [298.454618] hardirqs last enabled at (1063853): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.454621] hardirqs last disabled at (1063858): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.454623] softirqs last enabled at (1063686): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.454626] softirqs last disabled at (1063671): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.454628] ---[ end trace 0000000000000000 ]--- <7> [298.454643] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.454749] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <6> [298.454848] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.454919] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.454925] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.455136] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.455343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.455437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.455532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.455622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.455712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.455799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.455887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.455974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.456055] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.456154] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.456271] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.457293] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.467431] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.467682] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.468675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.468755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.468832] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.468910] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.468985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.469060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.469135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.469210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.469284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.469358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.469438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.469513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.469588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.469661] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.469735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.469809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.469886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.469962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.470037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.470118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.470198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.470276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.470355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.470440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.470521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.470600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.470677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.470756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.470835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.470914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.470993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.471073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.471157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.471240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.471324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.471404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.471501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.471582] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.471663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.471742] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.471825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.471904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.471982] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.472060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.472141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.472218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.472295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.472370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.472457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.472537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.472614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.472695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.472774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.472853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.472929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.473007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.473082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.473157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.473236] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.473347] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.473351] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41376, lrc_seqno=41376, guc_id=0, flags=0x73 in no process [-1] <7> [298.473353] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.473416] ------------[ cut here ]------------ <4> [298.473418] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.473419] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.473492] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.473556] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.473564] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.473567] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.473568] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.473569] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.473574] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.473643] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.473645] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.473647] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.473648] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.473650] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.473651] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.473652] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.473653] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.473655] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.473656] CR2: 000072fbb0ae1000 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [298.473658] PKRU: 55555554 <4> [298.473659] Call Trace: <4> [298.473660] <TASK> <4> [298.473664] ? lock_sync+0x100/0x100 <4> [298.473668] ? lock_release+0xd0/0x2b0 <4> [298.473674] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.473679] process_one_work+0x239/0x760 <4> [298.473685] worker_thread+0x200/0x3f0 <4> [298.473688] ? __pfx_worker_thread+0x10/0x10 <4> [298.473690] kthread+0x10d/0x150 <4> [298.473693] ? __pfx_kthread+0x10/0x10 <4> [298.473697] ret_from_fork+0x3d4/0x480 <4> [298.473699] ? __pfx_kthread+0x10/0x10 <4> [298.473702] ret_from_fork_asm+0x1a/0x30 <4> [298.473710] </TASK> <4> [298.473711] irq event stamp: 859815 <4> [298.473712] hardirqs last enabled at (859821): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.473715] hardirqs last disabled at (859826): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.473717] softirqs last enabled at (858876): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.473720] softirqs last disabled at (858869): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.473722] ---[ end trace 0000000000000000 ]--- <6> [298.473723] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.473792] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.473798] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.475031] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.475136] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.475240] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.475447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.475533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.475626] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.475714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.475804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.475891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.475978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.476065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.476147] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.476240] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.476356] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.477364] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.488093] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.488336] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.489483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.489554] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.489624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.489696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.489765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.489834] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.489904] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.489975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.490046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.490115] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.490184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.490252] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.490320] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.490388] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.490467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.490542] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.490616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.490690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.490764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.490845] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.490926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.491007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.491087] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.491165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.491241] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.491316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.491393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.491481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.491561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.491639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.491717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.491795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.491880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.491966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.492053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.492135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.492214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.492291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.492367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.492452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.492531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.492609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.492685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.492760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.492839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.492915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.492991] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.493066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.493144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.493219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.493294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.493373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.493460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.493538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.493614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.493696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.493776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.493854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.493935] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.494047] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.494051] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41376, lrc_seqno=41376, guc_id=0, flags=0x73 in no process [-1] <7> [298.494053] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.494111] ------------[ cut here ]------------ <4> [298.494112] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.494114] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.494186] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.494249] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.494257] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.494260] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.494261] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.494263] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.494268] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.494337] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.494339] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.494341] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.494343] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.494344] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.494345] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.494346] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.494348] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.494349] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.494351] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.494352] PKRU: 55555554 <4> [298.494353] Call Trace: <4> [298.494354] <TASK> <4> [298.494358] ? lock_sync+0x100/0x100 <4> [298.494363] ? lock_release+0xd0/0x2b0 <4> [298.494368] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.494374] process_one_work+0x239/0x760 <4> [298.494380] worker_thread+0x200/0x3f0 <4> [298.494383] ? __pfx_worker_thread+0x10/0x10 <4> [298.494385] kthread+0x10d/0x150 <4> [298.494388] ? __pfx_kthread+0x10/0x10 <4> [298.494391] ret_from_fork+0x3d4/0x480 <4> [298.494393] ? __pfx_kthread+0x10/0x10 <4> [298.494397] ret_from_fork_asm+0x1a/0x30 <4> [298.494404] </TASK> <4> [298.494405] irq event stamp: 1907039 <4> [298.494406] hardirqs last enabled at (1907045): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.494412] hardirqs last disabled at (1907052): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.494415] softirqs last enabled at (1906128): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.494417] softirqs last disabled at (1906117): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.494420] ---[ end trace 0000000000000000 ]--- <5> [298.495780] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41431, lrc_seqno=41431, guc_id=0, flags=0x73 in no process [-1] <7> [298.495783] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.495845] ------------[ cut here ]------------ <4> [298.495847] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.495848] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.495919] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.495982] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.495990] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.495993] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.495994] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.495995] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.496000] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.496069] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.496070] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.496072] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.496074] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.496075] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.496076] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.496077] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.496078] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.496080] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.496081] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.496082] PKRU: 55555554 <4> [298.496083] Call Trace: <4> [298.496085] <TASK> <4> [298.496088] ? lock_sync+0x100/0x100 <4> [298.496092] ? lock_release+0xd0/0x2b0 <4> [298.496098] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.496103] process_one_work+0x239/0x760 <4> [298.496109] worker_thread+0x200/0x3f0 <4> [298.496112] ? __pfx_worker_thread+0x10/0x10 <4> [298.496114] kthread+0x10d/0x150 <4> [298.496117] ? __pfx_kthread+0x10/0x10 <4> [298.496121] ret_from_fork+0x3d4/0x480 <4> [298.496123] ? __pfx_kthread+0x10/0x10 <4> [298.496126] ret_from_fork_asm+0x1a/0x30 <4> [298.496133] </TASK> <4> [298.496134] irq event stamp: 1908953 <4> [298.496135] hardirqs last enabled at (1908959): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.496138] hardirqs last disabled at (1908964): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.496140] softirqs last enabled at (1906128): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.496142] softirqs last disabled at (1906117): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.496144] ---[ end trace 0000000000000000 ]--- <6> [298.496146] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.496215] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.496221] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.496461] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.496564] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.496671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.496874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.496963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.497056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.497145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.497233] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.497319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.497405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.497510] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.497593] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.497689] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.497807] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.498815] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.509417] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.509669] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.510676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.510755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.510830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.510907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.510986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.511062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.511137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.511213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.511287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.511361] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.511442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.511516] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.511589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.511663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.511737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.511812] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.511886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.511959] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.512033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.512116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.512198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.512277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.512357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.512439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.512516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.512593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.512669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.512749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.512832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.512914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.512994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.513074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.513159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.513242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.513325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.513406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.513500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.513578] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.513655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.513735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.513817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.513897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.513976] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.514053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.514135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.514212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.514289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.514365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.514451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.514527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.514603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.514681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.514759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.514838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.514912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.514990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.515067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.515147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.515230] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.515342] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.515346] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41431, lrc_seqno=41431, guc_id=0, flags=0x73 in no process [-1] <7> [298.515348] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.515406] ------------[ cut here ]------------ <4> [298.515411] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.515412] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.515483] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.515547] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.515555] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.515558] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.515559] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.515560] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.515565] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.515634] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.515635] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.515637] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.515639] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.515640] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.515641] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.515642] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.515644] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.515645] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.515647] CR2: 00007f74cf18b008 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.515648] PKRU: 55555554 <4> [298.515649] Call Trace: <4> [298.515650] <TASK> <4> [298.515654] ? lock_sync+0x100/0x100 <4> [298.515659] ? lock_release+0xd0/0x2b0 <4> [298.515664] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.515670] process_one_work+0x239/0x760 <4> [298.515676] worker_thread+0x200/0x3f0 <4> [298.515679] ? __pfx_worker_thread+0x10/0x10 <4> [298.515681] kthread+0x10d/0x150 <4> [298.515684] ? __pfx_kthread+0x10/0x10 <4> [298.515688] ret_from_fork+0x3d4/0x480 <4> [298.515690] ? __pfx_kthread+0x10/0x10 <4> [298.515693] ret_from_fork_asm+0x1a/0x30 <4> [298.515700] </TASK> <4> [298.515701] irq event stamp: 1912043 <4> [298.515702] hardirqs last enabled at (1912049): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.515705] hardirqs last disabled at (1912054): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.515707] softirqs last enabled at (1911104): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.515710] softirqs last disabled at (1911097): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.515712] ---[ end trace 0000000000000000 ]--- <6> [298.516637] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.516709] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.516716] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.516730] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.516834] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.516939] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.517154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.517247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.517342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.517437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.517524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.517612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.517701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.517789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.517872] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.517968] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.518084] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.519118] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.529417] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.529665] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.530854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.530933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.531002] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.531073] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.531143] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.531213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.531284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.531354] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.531430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.531506] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.531580] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.531655] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.531730] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.531803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.531876] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.531950] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.532024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.532098] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.532172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.532252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.532334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.532422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.532502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.532580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.532659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.532737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.532819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.532901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.532986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.533070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.533150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.533228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.533312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.533397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.533517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.533645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.533731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.533811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.533890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.533966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.534045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.534124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.534201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.534278] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.534359] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.534444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.534522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.534599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.534677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.534754] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.534834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.534917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.534997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.535075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.535152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.535230] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.535306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.535382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.535475] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.535588] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.535593] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41431, lrc_seqno=41431, guc_id=0, flags=0x73 in no process [-1] <7> [298.535595] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.535653] ------------[ cut here ]------------ <4> [298.535654] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.535656] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#7: kworker/u64:14/2466 <4> [298.535728] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.535793] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.535801] CPU: 7 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.535804] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.535805] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.535807] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.535812] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.535882] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.535884] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.535886] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.535888] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.535889] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.535890] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.535891] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.535893] FS: 0000000000000000(0000) GS:ffff8888db017000(0000) knlGS:0000000000000000 <4> [298.535894] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.535896] CR2: 000072fbb0ae5000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.535897] PKRU: 55555554 <4> [298.535898] Call Trace: <4> [298.535900] <TASK> <4> [298.535903] ? lock_sync+0x100/0x100 <4> [298.535908] ? lock_release+0xd0/0x2b0 <4> [298.535914] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.535919] process_one_work+0x239/0x760 <4> [298.535926] worker_thread+0x200/0x3f0 <4> [298.535929] ? __pfx_worker_thread+0x10/0x10 <4> [298.535931] kthread+0x10d/0x150 <4> [298.535934] ? __pfx_kthread+0x10/0x10 <4> [298.535938] ret_from_fork+0x3d4/0x480 <4> [298.535940] ? __pfx_kthread+0x10/0x10 <4> [298.535943] ret_from_fork_asm+0x1a/0x30 <4> [298.535951] </TASK> <4> [298.535952] irq event stamp: 1070031 <4> [298.535953] hardirqs last enabled at (1070037): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.535956] hardirqs last disabled at (1070042): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.535958] softirqs last enabled at (1069158): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.535961] softirqs last disabled at (1069137): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.535963] ---[ end trace 0000000000000000 ]--- <5> [298.536551] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41486, lrc_seqno=41486, guc_id=0, flags=0x73 in no process [-1] <7> [298.536560] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.536935] ------------[ cut here ]------------ <4> [298.536938] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.536941] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:14/2466 <4> [298.537311] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.537387] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.537395] CPU: 6 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.537398] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.537399] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.537401] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.537406] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <7> [298.537433] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [298.537544] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.537547] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.537550] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.537552] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.537554] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.537556] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.537558] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.537560] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [298.537562] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.537564] CR2: 000077f1a4387048 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [298.537566] PKRU: 55555554 <4> [298.537568] Call Trace: <4> [298.537569] <TASK> <4> [298.537576] ? lock_sync+0x100/0x100 <4> [298.537584] ? lock_release+0xd0/0x2b0 <4> [298.537593] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.537603] process_one_work+0x239/0x760 <4> [298.537613] worker_thread+0x200/0x3f0 <4> [298.537617] ? __pfx_worker_thread+0x10/0x10 <4> [298.537620] kthread+0x10d/0x150 <4> [298.537624] ? __pfx_kthread+0x10/0x10 <4> [298.537630] ret_from_fork+0x3d4/0x480 <4> [298.537633] ? __pfx_kthread+0x10/0x10 <4> [298.537638] ret_from_fork_asm+0x1a/0x30 <4> [298.537652] </TASK> <4> [298.537653] irq event stamp: 1071941 <4> [298.537655] hardirqs last enabled at (1071947): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.537659] hardirqs last disabled at (1071952): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <7> [298.537590] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [298.537663] softirqs last enabled at (1071780): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.537666] softirqs last disabled at (1071773): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.537669] ---[ end trace 0000000000000000 ]--- <6> [298.537672] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.537769] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.537776] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.538014] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.538217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.538307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.538402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.538511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.538597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.538682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.538769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.538856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.538939] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.539035] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.539151] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.540160] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.550415] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.550667] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.551668] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.551748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.551825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.551902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.551979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.552058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.552136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.552212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.552288] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.552363] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.552447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.552521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.552596] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.552669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.552743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.552818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.552890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.552963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.553037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.553119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.553201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.553279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.553360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.553446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.553527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.553607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.553685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.553764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.553844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.553921] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.554001] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.554079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.554162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.554246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.554329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.554414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.554494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.554571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.554649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.554729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.554811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.554890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.554970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.555048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.555128] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.555206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.555283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.555360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.555448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.555525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.555606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.555689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.555770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.555850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.555928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.556007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.556085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.556170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.556251] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.556650] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.556655] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41486, lrc_seqno=41486, guc_id=0, flags=0x73 in no process [-1] <7> [298.556658] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.556719] ------------[ cut here ]------------ <4> [298.556721] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.556722] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:14/2466 <4> [298.556795] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.556860] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.556868] CPU: 6 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.556871] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.556873] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.556874] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.556879] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.556950] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.556952] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.556954] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.556956] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.556957] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.556958] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.556959] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.556961] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [298.556962] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.556964] CR2: 000077f1a4387048 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [298.556965] PKRU: 55555554 <4> [298.556966] Call Trace: <4> [298.556967] <TASK> <4> [298.556971] ? lock_sync+0x100/0x100 <4> [298.556976] ? lock_release+0xd0/0x2b0 <4> [298.556981] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.556987] process_one_work+0x239/0x760 <4> [298.556993] worker_thread+0x200/0x3f0 <4> [298.556996] ? __pfx_worker_thread+0x10/0x10 <4> [298.556998] kthread+0x10d/0x150 <4> [298.557001] ? __pfx_kthread+0x10/0x10 <4> [298.557005] ret_from_fork+0x3d4/0x480 <4> [298.557007] ? __pfx_kthread+0x10/0x10 <4> [298.557010] ret_from_fork_asm+0x1a/0x30 <4> [298.557017] </TASK> <4> [298.557019] irq event stamp: 1075049 <4> [298.557020] hardirqs last enabled at (1075055): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.557023] hardirqs last disabled at (1075060): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.557025] softirqs last enabled at (1074222): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.557027] softirqs last disabled at (1074215): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.557030] ---[ end trace 0000000000000000 ]--- <6> [298.557031] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.557101] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.557107] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.558352] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.558469] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.558575] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.558782] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.558875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.558981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.559072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.559161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.559251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.559339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.559433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.559517] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.559613] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.559731] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.560737] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.571417] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.571664] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.572811] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.572892] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.572962] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.573034] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.573105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.573175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.573245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.573314] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.573383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.573465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.573545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.573622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.573699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.573776] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.573851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.573926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.574001] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.574077] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.574154] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.574236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.574317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.574395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.574488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.574565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.574643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.574722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.574802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.574883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.574964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.575043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.575122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.575201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.575284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.575368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.575463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.575547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.575630] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.575709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.575787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.575863] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.575946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.576029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.576109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.576187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.576267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.576344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.576428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.576505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.576584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.576660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.576739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.576822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.576901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.576979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.577056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.577134] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.577210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.577285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.577365] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.577485] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.577489] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41486, lrc_seqno=41486, guc_id=0, flags=0x73 in no process [-1] <7> [298.577492] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.577550] ------------[ cut here ]------------ <4> [298.577551] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.577552] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.577623] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.577688] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.577696] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.577699] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.577700] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.577702] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.577707] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.577775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.577777] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.577779] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.577781] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.577782] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.577783] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.577785] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.577786] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.577788] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.577789] CR2: 000072fbb0ae6000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.577790] PKRU: 55555554 <4> [298.577792] Call Trace: <4> [298.577793] <TASK> <4> [298.577797] ? lock_sync+0x100/0x100 <4> [298.577801] ? lock_release+0xd0/0x2b0 <4> [298.577807] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.577812] process_one_work+0x239/0x760 <4> [298.577818] worker_thread+0x200/0x3f0 <4> [298.577821] ? __pfx_worker_thread+0x10/0x10 <4> [298.577824] kthread+0x10d/0x150 <4> [298.577826] ? __pfx_kthread+0x10/0x10 <4> [298.577830] ret_from_fork+0x3d4/0x480 <4> [298.577832] ? __pfx_kthread+0x10/0x10 <4> [298.577836] ret_from_fork_asm+0x1a/0x30 <4> [298.577843] </TASK> <4> [298.577844] irq event stamp: 1915671 <4> [298.577845] hardirqs last enabled at (1915677): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.577848] hardirqs last disabled at (1915682): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.577850] softirqs last enabled at (1914806): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.577853] softirqs last disabled at (1914795): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.577855] ---[ end trace 0000000000000000 ]--- <5> [298.579073] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41541, lrc_seqno=41541, guc_id=0, flags=0x73 in no process [-1] <7> [298.579076] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.579137] ------------[ cut here ]------------ <4> [298.579138] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.579140] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.579212] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.579275] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.579283] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.579285] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.579286] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.579288] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.579292] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.579362] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.579364] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.579366] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.579367] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.579368] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.579370] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.579371] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.579372] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.579374] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.579375] CR2: 000072fbb0ae6000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.579376] PKRU: 55555554 <4> [298.579377] Call Trace: <4> [298.579378] <TASK> <4> [298.579382] ? lock_sync+0x100/0x100 <4> [298.579386] ? lock_release+0xd0/0x2b0 <4> [298.579392] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.579397] process_one_work+0x239/0x760 <4> [298.579403] worker_thread+0x200/0x3f0 <4> [298.579410] ? __pfx_worker_thread+0x10/0x10 <4> [298.579412] kthread+0x10d/0x150 <4> [298.579415] ? __pfx_kthread+0x10/0x10 <4> [298.579419] ret_from_fork+0x3d4/0x480 <4> [298.579421] ? __pfx_kthread+0x10/0x10 <4> [298.579425] ret_from_fork_asm+0x1a/0x30 <4> [298.579432] </TASK> <4> [298.579433] irq event stamp: 1917567 <4> [298.579434] hardirqs last enabled at (1917573): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.579437] hardirqs last disabled at (1917578): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.579439] softirqs last enabled at (1914806): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.579442] softirqs last disabled at (1914795): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.579444] ---[ end trace 0000000000000000 ]--- <6> [298.579445] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.579515] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.579521] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.579675] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.579779] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.579885] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.580092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.580183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.580276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.580365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.580461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.580548] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.580635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.580723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.580809] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.580907] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.581026] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.582032] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.592414] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.592664] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.593669] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.593748] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.593826] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.593905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.593982] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.594058] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.594133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.594208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.594282] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.594355] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.594434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.594508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.594581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.594657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.594734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.594810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.594884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.594958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.595032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.595113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.595194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.595274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.595355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.595440] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.595518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.595594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.595670] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.595749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.595828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.595906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.595984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.596061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.596144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.596227] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.596311] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.596391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.596480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.596556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.596634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.596710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.596794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.596875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.596954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.597031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.597111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.597189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.597266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.597342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.597422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.597501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.597581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.597662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.597741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.597819] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.597896] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.597974] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.598049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.598125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.598204] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.598316] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.598320] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41541, lrc_seqno=41541, guc_id=0, flags=0x73 in no process [-1] <7> [298.598323] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.598382] ------------[ cut here ]------------ <4> [298.598383] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.598384] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:14/2466 <4> [298.598467] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.598531] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.598540] CPU: 6 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.598542] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.598543] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.598545] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.598550] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.598620] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.598622] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.598624] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.598626] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.598627] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.598628] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.598629] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.598631] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [298.598632] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.598634] CR2: 000077f1a4387048 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.598635] PKRU: 55555554 <4> [298.598637] Call Trace: <4> [298.598638] <TASK> <4> [298.598641] ? lock_sync+0x100/0x100 <4> [298.598646] ? lock_release+0xd0/0x2b0 <4> [298.598651] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.598657] process_one_work+0x239/0x760 <4> [298.598663] worker_thread+0x200/0x3f0 <4> [298.598666] ? __pfx_worker_thread+0x10/0x10 <4> [298.598668] kthread+0x10d/0x150 <4> [298.598671] ? __pfx_kthread+0x10/0x10 <4> [298.598675] ret_from_fork+0x3d4/0x480 <4> [298.598677] ? __pfx_kthread+0x10/0x10 <4> [298.598680] ret_from_fork_asm+0x1a/0x30 <4> [298.598687] </TASK> <4> [298.598688] irq event stamp: 1078643 <4> [298.598690] hardirqs last enabled at (1078649): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.598692] hardirqs last disabled at (1078654): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.598695] softirqs last enabled at (1077870): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.598697] softirqs last disabled at (1077855): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.598699] ---[ end trace 0000000000000000 ]--- <6> [298.598701] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.598771] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.598778] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.600017] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.600122] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.600226] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.600439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.600532] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.600628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.600718] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.600809] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.600898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.600986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.601076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.601159] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.601256] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.601375] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.602389] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.613129] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.613378] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.614536] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.614616] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.614694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.614771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.614849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.614925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.615000] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.615076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.615152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.615229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.615308] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.615384] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.615470] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.615544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.615619] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.615694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.615769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.615843] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.615919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.616005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.616089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.616171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.616252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.616331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.616412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.616490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.616568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.616648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.616731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.616810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.616890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.616971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.617061] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.617149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.617236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.617319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.617403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.617499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.617579] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.617659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.617741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.617823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.617906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.617986] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.618069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.618147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.618226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.618303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.618382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.618469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.618545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.618629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.618712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.618792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.618872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.618956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.619035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.619115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.619196] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.619308] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.619312] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41541, lrc_seqno=41541, guc_id=0, flags=0x73 in no process [-1] <7> [298.619315] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.619374] ------------[ cut here ]------------ <4> [298.619375] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.619376] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.619457] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.619522] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.619530] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.619533] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.619535] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.619536] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.619541] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.619612] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.619613] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.619616] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.619617] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.619618] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.619619] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.619621] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.619622] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.619624] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.619625] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.619627] PKRU: 55555554 <4> [298.619628] Call Trace: <4> [298.619629] <TASK> <4> [298.619633] ? lock_sync+0x100/0x100 <4> [298.619638] ? lock_release+0xd0/0x2b0 <4> [298.619643] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.619649] process_one_work+0x239/0x760 <4> [298.619655] worker_thread+0x200/0x3f0 <4> [298.619658] ? __pfx_worker_thread+0x10/0x10 <4> [298.619660] kthread+0x10d/0x150 <4> [298.619663] ? __pfx_kthread+0x10/0x10 <4> [298.619667] ret_from_fork+0x3d4/0x480 <4> [298.619669] ? __pfx_kthread+0x10/0x10 <4> [298.619672] ret_from_fork_asm+0x1a/0x30 <4> [298.619680] </TASK> <4> [298.619681] irq event stamp: 866009 <4> [298.619682] hardirqs last enabled at (866015): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.619685] hardirqs last disabled at (866020): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.619687] softirqs last enabled at (865076): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.619689] softirqs last disabled at (865069): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.619692] ---[ end trace 0000000000000000 ]--- <5> [298.620937] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41596, lrc_seqno=41596, guc_id=0, flags=0x73 in no process [-1] <7> [298.620941] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.621009] ------------[ cut here ]------------ <4> [298.621010] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.621012] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.621082] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.621145] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.621153] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.621155] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.621156] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.621158] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.621162] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.621231] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.621232] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.621234] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.621236] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.621237] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.621238] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.621239] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.621241] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.621242] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.621243] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.621245] PKRU: 55555554 <4> [298.621246] Call Trace: <4> [298.621247] <TASK> <4> [298.621251] ? lock_sync+0x100/0x100 <4> [298.621255] ? lock_release+0xd0/0x2b0 <4> [298.621260] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.621266] process_one_work+0x239/0x760 <4> [298.621272] worker_thread+0x200/0x3f0 <4> [298.621274] ? __pfx_worker_thread+0x10/0x10 <4> [298.621277] kthread+0x10d/0x150 <4> [298.621280] ? __pfx_kthread+0x10/0x10 <4> [298.621283] ret_from_fork+0x3d4/0x480 <4> [298.621285] ? __pfx_kthread+0x10/0x10 <4> [298.621289] ret_from_fork_asm+0x1a/0x30 <4> [298.621296] </TASK> <4> [298.621297] irq event stamp: 867927 <4> [298.621298] hardirqs last enabled at (867933): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.621300] hardirqs last disabled at (867938): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.621303] softirqs last enabled at (865076): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.621305] softirqs last disabled at (865069): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.621307] ---[ end trace 0000000000000000 ]--- <6> [298.621309] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.621379] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.621385] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.621602] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.621702] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.621808] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.622010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.622100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.622194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.622281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.622369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.622465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.622551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.622637] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.622719] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.622812] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.622930] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.623938] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.634413] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.634662] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.635671] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.635750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.635827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.635905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.635981] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.636057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.636133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.636209] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.636285] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.636362] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.636447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.636524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.636599] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.636674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.636747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.636821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.636894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.636967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.637041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.637122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.637203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.637283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.637364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.637453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.637535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.637614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.637693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.637772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.637852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.637931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.638011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.638089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.638174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.638258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.638345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.638433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.638516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.638596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.638674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.638750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.638836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.638918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.638998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.639076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.639157] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.639234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.639310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.639387] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.639481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.639562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.639640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.639725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.639807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.639887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.639964] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.640043] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.640120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.640197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.640276] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.640386] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.640390] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41596, lrc_seqno=41596, guc_id=0, flags=0x73 in no process [-1] <7> [298.640393] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.641318] ------------[ cut here ]------------ <4> [298.641320] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.641321] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:13/2465 <4> [298.641395] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.641463] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.641471] CPU: 8 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.641474] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.641475] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.641477] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.641482] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.641553] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.641555] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.641557] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.641558] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.641559] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.641561] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.641562] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.641563] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.641565] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.641566] CR2: 000072fbb0ae1000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.641567] PKRU: 55555554 <4> [298.641569] Call Trace: <4> [298.641570] <TASK> <4> [298.641574] ? lock_sync+0x100/0x100 <4> [298.641578] ? lock_release+0xd0/0x2b0 <4> [298.641583] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.641589] process_one_work+0x239/0x760 <4> [298.641595] worker_thread+0x200/0x3f0 <4> [298.641598] ? __pfx_worker_thread+0x10/0x10 <4> [298.641600] kthread+0x10d/0x150 <4> [298.641603] ? __pfx_kthread+0x10/0x10 <4> [298.641607] ret_from_fork+0x3d4/0x480 <4> [298.641609] ? __pfx_kthread+0x10/0x10 <4> [298.641612] ret_from_fork_asm+0x1a/0x30 <4> [298.641620] </TASK> <4> [298.641621] irq event stamp: 871037 <4> [298.641622] hardirqs last enabled at (871043): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.641625] hardirqs last disabled at (871048): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.641627] softirqs last enabled at (870092): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.641629] softirqs last disabled at (870085): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.641632] ---[ end trace 0000000000000000 ]--- <6> [298.641633] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.641703] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.641709] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.641723] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.641828] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.642115] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.642323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.642421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.642517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.642606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.642692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.642777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.642862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.642949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.643030] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.643126] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.643244] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.644253] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.654413] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.654662] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.655758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.655837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.655913] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.655992] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.656069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.656148] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.656227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.656305] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.656381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.656465] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.656540] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.656615] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.656689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.656763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.656839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.656915] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.656990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.657065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.657141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.657222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.657303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.657384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.657474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.657555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.657636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.657715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.657792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.657872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.657952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.658033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.658115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.658197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.658282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.658367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.658459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.658541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.658620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.658698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.658775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.658851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.658935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.659018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.659099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.659176] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.659259] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.659337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.659421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.659501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.659581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.659657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.659733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.659812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.659892] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.659973] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.660051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.660129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.660205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.660282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.660365] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.660490] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.660499] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41596, lrc_seqno=41596, guc_id=0, flags=0x73 in no process [-1] <7> [298.660501] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.660560] ------------[ cut here ]------------ <4> [298.660561] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.660562] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.660635] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.660699] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.660707] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.660710] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.660711] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.660713] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.660718] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.660788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.660789] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.660792] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.660793] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.660794] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.660796] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.660797] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.660798] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.660800] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.660801] CR2: 000072fbb0ae6000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.660803] PKRU: 55555554 <4> [298.660804] Call Trace: <4> [298.660805] <TASK> <4> [298.660809] ? lock_sync+0x100/0x100 <4> [298.660813] ? lock_release+0xd0/0x2b0 <4> [298.660818] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.660824] process_one_work+0x239/0x760 <4> [298.660830] worker_thread+0x200/0x3f0 <4> [298.660833] ? __pfx_worker_thread+0x10/0x10 <4> [298.660835] kthread+0x10d/0x150 <4> [298.660838] ? __pfx_kthread+0x10/0x10 <4> [298.660842] ret_from_fork+0x3d4/0x480 <4> [298.660844] ? __pfx_kthread+0x10/0x10 <4> [298.660847] ret_from_fork_asm+0x1a/0x30 <4> [298.660855] </TASK> <4> [298.660856] irq event stamp: 1922597 <4> [298.660857] hardirqs last enabled at (1922603): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.660860] hardirqs last disabled at (1922608): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.660862] softirqs last enabled at (1921730): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.660865] softirqs last disabled at (1921715): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.660867] ---[ end trace 0000000000000000 ]--- <5> [298.661227] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41651, lrc_seqno=41651, guc_id=0, flags=0x73 in no process [-1] <7> [298.661234] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.661476] ------------[ cut here ]------------ <4> [298.661479] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.661483] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:13/2465 <4> [298.661604] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.661704] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.661717] CPU: 12 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.661721] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.661723] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.661726] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.661735] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.661850] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.661853] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.661857] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.661859] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.661861] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.661863] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.661865] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.661867] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [298.661869] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.661871] CR2: 000072fbb0ae2000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [298.661873] PKRU: 55555554 <4> [298.661875] Call Trace: <4> [298.661877] <TASK> <4> [298.661883] ? lock_sync+0x100/0x100 <4> [298.661891] ? lock_release+0xd0/0x2b0 <4> [298.661900] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.661909] process_one_work+0x239/0x760 <4> [298.661920] worker_thread+0x200/0x3f0 <4> [298.661924] ? __pfx_worker_thread+0x10/0x10 <4> [298.661928] kthread+0x10d/0x150 <4> [298.661932] ? __pfx_kthread+0x10/0x10 <4> [298.661937] ret_from_fork+0x3d4/0x480 <4> [298.661941] ? __pfx_kthread+0x10/0x10 <4> [298.661946] ret_from_fork_asm+0x1a/0x30 <4> [298.661957] </TASK> <4> [298.661959] irq event stamp: 872539 <4> [298.661961] hardirqs last enabled at (872545): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.661966] hardirqs last disabled at (872550): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.661969] softirqs last enabled at (871748): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.661972] softirqs last disabled at (871743): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.661975] ---[ end trace 0000000000000000 ]--- <6> [298.661978] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.662092] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.662101] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.662176] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.662282] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.662390] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.662607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.662695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.662788] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.662875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.662961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.663046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.663131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.663217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.663298] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.663393] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.663523] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.664533] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.674413] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.674663] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.675624] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.675702] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.675780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.675859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.675937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.676015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.676091] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.676167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.676242] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.676317] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.676392] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.676478] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.676553] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.676627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.676700] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.676774] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.676849] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.676925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.677002] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.677084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.677165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.677247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.677329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.677412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.677491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.677568] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.677645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.677723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.677802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.677880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.677958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.678036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.678119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.678202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.678286] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.678367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.678455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.678531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.678606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.678683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.678767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.678848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.678927] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.679004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.679085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.679161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.679239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.679317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.679394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.679483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.679563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.679647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.679727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.679805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.679883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.679966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.680045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.680123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.680205] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.680317] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.680321] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41651, lrc_seqno=41651, guc_id=0, flags=0x73 in no process [-1] <7> [298.680324] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.680382] ------------[ cut here ]------------ <4> [298.680383] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.680385] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.680466] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.680531] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.680539] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.680541] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.680542] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.680544] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.680549] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.680619] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.680621] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.680623] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.680624] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.680626] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.680627] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.680628] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.680630] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.680631] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.680632] CR2: 000072fbb0ae6000 CR3: 0000000133b90001 CR4: 0000000000f72ef0 <4> [298.680634] PKRU: 55555554 <4> [298.680635] Call Trace: <4> [298.680636] <TASK> <4> [298.680640] ? lock_sync+0x100/0x100 <4> [298.680644] ? lock_release+0xd0/0x2b0 <4> [298.680650] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.680655] process_one_work+0x239/0x760 <4> [298.680661] worker_thread+0x200/0x3f0 <4> [298.680664] ? __pfx_worker_thread+0x10/0x10 <4> [298.680667] kthread+0x10d/0x150 <4> [298.680670] ? __pfx_kthread+0x10/0x10 <4> [298.680673] ret_from_fork+0x3d4/0x480 <4> [298.680675] ? __pfx_kthread+0x10/0x10 <4> [298.680679] ret_from_fork_asm+0x1a/0x30 <4> [298.680686] </TASK> <4> [298.680687] irq event stamp: 1926851 <4> [298.680688] hardirqs last enabled at (1926857): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.680691] hardirqs last disabled at (1926862): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.680693] softirqs last enabled at (1925918): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.680696] softirqs last disabled at (1925911): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.680698] ---[ end trace 0000000000000000 ]--- <6> [298.680700] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.680770] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.680776] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.681183] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.681384] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.681480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.681572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.681659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.681745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.681830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.681919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.682011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.682095] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.682190] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.682307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.683312] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.694033] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.694274] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.695336] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.695421] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.695496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.695567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.695636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.695705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.695773] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.695841] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.695908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.695977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.696048] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.696118] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.696187] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.696255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.696324] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.696393] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.696480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.696556] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.696631] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.696712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.696793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.696873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.696955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.697034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.697111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.697187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.697263] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.697342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.697424] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.697502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.697581] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.697659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.697743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.697827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.697914] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.697998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.698080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.698159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.698237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.698314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.698394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.698484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.698562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.698640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.698721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.698802] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.698883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.698962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.699040] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.699117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.699193] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.699273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.699351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.699433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.699508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.699586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.699662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.699737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.699816] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.699925] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.699929] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41651, lrc_seqno=41651, guc_id=0, flags=0x73 in no process [-1] <7> [298.699931] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.699990] ------------[ cut here ]------------ <4> [298.699991] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.699993] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.700065] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.700129] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.700137] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.700140] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.700141] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.700142] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.700147] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.700217] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.700218] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.700221] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.700222] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.700223] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.700225] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.700226] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.700227] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.700229] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.700230] CR2: 000072fbb0ae6000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.700232] PKRU: 55555554 <4> [298.700233] Call Trace: <4> [298.700234] <TASK> <4> [298.700238] ? lock_sync+0x100/0x100 <4> [298.700242] ? lock_release+0xd0/0x2b0 <4> [298.700247] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.700253] process_one_work+0x239/0x760 <4> [298.700259] worker_thread+0x200/0x3f0 <4> [298.700262] ? __pfx_worker_thread+0x10/0x10 <4> [298.700264] kthread+0x10d/0x150 <4> [298.700267] ? __pfx_kthread+0x10/0x10 <4> [298.700271] ret_from_fork+0x3d4/0x480 <4> [298.700273] ? __pfx_kthread+0x10/0x10 <4> [298.700276] ret_from_fork_asm+0x1a/0x30 <4> [298.700283] </TASK> <4> [298.700285] irq event stamp: 1929951 <4> [298.700286] hardirqs last enabled at (1929957): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.700288] hardirqs last disabled at (1929962): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.700291] softirqs last enabled at (1928856): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.700293] softirqs last disabled at (1928849): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.700295] ---[ end trace 0000000000000000 ]--- <5> [298.701799] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41706, lrc_seqno=41706, guc_id=0, flags=0x73 in no process [-1] <7> [298.701803] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [298.701809] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [298.701866] ------------[ cut here ]------------ <4> [298.701867] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.701869] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:14/2466 <4> [298.701943] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers <7> [298.701914] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [298.701962] i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.702022] CPU: 8 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.702025] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.702026] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.702027] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.702032] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.702104] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.702105] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.702108] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.702109] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.702110] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.702111] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.702113] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.702114] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.702116] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.702117] CR2: 000072fbb0ae8000 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [298.702118] PKRU: 55555554 <4> [298.702120] Call Trace: <4> [298.702121] <TASK> <4> [298.702125] ? lock_sync+0x100/0x100 <4> [298.702129] ? lock_release+0xd0/0x2b0 <4> [298.702134] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.702140] process_one_work+0x239/0x760 <4> [298.702146] worker_thread+0x200/0x3f0 <4> [298.702149] ? __pfx_worker_thread+0x10/0x10 <4> [298.702151] kthread+0x10d/0x150 <4> [298.702154] ? __pfx_kthread+0x10/0x10 <4> [298.702158] ret_from_fork+0x3d4/0x480 <4> [298.702160] ? __pfx_kthread+0x10/0x10 <4> [298.702163] ret_from_fork_asm+0x1a/0x30 <4> [298.702170] </TASK> <4> [298.702171] irq event stamp: 1082587 <4> [298.702173] hardirqs last enabled at (1082593): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.702176] hardirqs last disabled at (1082598): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.702178] softirqs last enabled at (1080906): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70 <4> [298.702180] softirqs last disabled at (1080904): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120 <4> [298.702183] ---[ end trace 0000000000000000 ]--- <6> [298.702185] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.702255] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.702262] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.702512] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.702773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.702911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.703050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.703184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.703317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.703456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.703590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.703725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.703856] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.703999] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.704165] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.705569] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.715415] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.715725] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.716882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.717004] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.717123] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.717244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.717365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.717495] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.717618] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.717738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.717859] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.717980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.718100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.718220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.718341] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.718467] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.718591] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.718712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.718833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.718953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.719074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.719198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.719324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.719462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.719558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.719644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.719728] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.719807] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.719886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.719967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.720048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.720130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.720212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.720296] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.720382] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.720478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.720562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.720641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.720720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.720797] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.720874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.720951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.721031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.721109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.721190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.721270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.721353] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.721437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.721517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.721594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.721673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.721748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.721825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.721906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.721987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.722067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.722147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.722226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.722302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.722378] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.722466] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.722579] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.722583] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41706, lrc_seqno=41706, guc_id=0, flags=0x73 in no process [-1] <7> [298.722586] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.722644] ------------[ cut here ]------------ <4> [298.722645] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.722646] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#6: kworker/u64:13/2465 <4> [298.722717] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.722782] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.722791] CPU: 6 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.722794] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.722795] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.722796] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.722801] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.722871] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.722873] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.722875] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.722877] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.722878] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.722879] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.722880] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.722882] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [298.722883] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.722885] CR2: 000077f1a4387048 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [298.722886] PKRU: 55555554 <4> [298.722887] Call Trace: <4> [298.722889] <TASK> <4> [298.722893] ? lock_sync+0x100/0x100 <4> [298.722897] ? lock_release+0xd0/0x2b0 <4> [298.722903] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.722908] process_one_work+0x239/0x760 <4> [298.722914] worker_thread+0x200/0x3f0 <4> [298.722917] ? __pfx_worker_thread+0x10/0x10 <4> [298.722920] kthread+0x10d/0x150 <4> [298.722922] ? __pfx_kthread+0x10/0x10 <4> [298.722926] ret_from_fork+0x3d4/0x480 <4> [298.722928] ? __pfx_kthread+0x10/0x10 <4> [298.722931] ret_from_fork_asm+0x1a/0x30 <4> [298.722939] </TASK> <4> [298.722940] irq event stamp: 876391 <4> [298.722941] hardirqs last enabled at (876397): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.722944] hardirqs last disabled at (876402): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.722946] softirqs last enabled at (875432): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.722949] softirqs last disabled at (875425): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.722951] ---[ end trace 0000000000000000 ]--- <6> [298.722952] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.723023] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.723029] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.724276] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.724382] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.724501] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.724704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.724795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.724890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.724981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.725069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.725156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.725242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.725330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.725432] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.725530] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.725649] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.726672] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.736410] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.736655] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.737785] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.737856] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.737926] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.737998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.738069] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.738138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.738208] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.738278] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.738349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.738428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.738505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.738574] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.738645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.738715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.738783] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.738851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.738920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.738988] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.739055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.739129] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.739203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.739276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.739349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.739425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.739507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.739580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.739651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.739724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.739798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.739870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.739941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.740013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.740090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.740167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.740244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.740317] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.740390] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.740474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.740547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.740616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.740691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.740765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.740837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.740907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.740981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.741052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.741122] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.741191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.741262] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.741331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.741399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.741489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.741565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.741639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.741714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.741792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.741862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.741931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.742006] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.742112] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.742116] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41706, lrc_seqno=41706, guc_id=0, flags=0x73 in no process [-1] <7> [298.742118] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.742177] ------------[ cut here ]------------ <4> [298.742178] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.742180] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#8: kworker/u64:14/2466 <4> [298.742245] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.742304] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.742311] CPU: 8 UID: 0 PID: 2466 Comm: kworker/u64:14 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.742314] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.742315] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.742316] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.742321] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.742389] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.742390] RSP: 0018:ffffc900042f7ca0 EFLAGS: 00010246 <4> [298.742393] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.742394] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.742395] RBP: ffffc900042f7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.742397] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.742398] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.742399] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [298.742401] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.742406] CR2: 000072fbb0ae8000 CR3: 0000000133b90002 CR4: 0000000000f72ef0 <4> [298.742407] PKRU: 55555554 <4> [298.742408] Call Trace: <4> [298.742410] <TASK> <4> [298.742414] ? lock_sync+0x100/0x100 <4> [298.742418] ? lock_release+0xd0/0x2b0 <4> [298.742423] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.742429] process_one_work+0x239/0x760 <4> [298.742436] worker_thread+0x200/0x3f0 <4> [298.742439] ? __pfx_worker_thread+0x10/0x10 <4> [298.742441] kthread+0x10d/0x150 <4> [298.742444] ? __pfx_kthread+0x10/0x10 <4> [298.742448] ret_from_fork+0x3d4/0x480 <4> [298.742450] ? __pfx_kthread+0x10/0x10 <4> [298.742453] ret_from_fork_asm+0x1a/0x30 <4> [298.742460] </TASK> <4> [298.742461] irq event stamp: 1086865 <4> [298.742463] hardirqs last enabled at (1086871): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.742465] hardirqs last disabled at (1086876): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.742468] softirqs last enabled at (1085944): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.742470] softirqs last disabled at (1085939): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.742473] ---[ end trace 0000000000000000 ]--- <5> [298.742839] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41761, lrc_seqno=41761, guc_id=0, flags=0x73 in no process [-1] <7> [298.742845] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.743050] ------------[ cut here ]------------ <4> [298.743052] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.743055] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.743135] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.743209] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.743218] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.743222] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.743223] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.743225] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.743233] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.743311] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.743313] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.743315] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.743317] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.743318] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.743319] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.743320] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.743322] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.743323] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.743324] CR2: 000072fbb0ae6000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.743326] PKRU: 55555554 <4> [298.743328] Call Trace: <4> [298.743329] <TASK> <4> [298.743334] ? lock_sync+0x100/0x100 <4> [298.743339] ? lock_release+0xd0/0x2b0 <4> [298.743346] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.743351] process_one_work+0x239/0x760 <4> [298.743358] worker_thread+0x200/0x3f0 <4> [298.743361] ? __pfx_worker_thread+0x10/0x10 <4> [298.743364] kthread+0x10d/0x150 <4> [298.743366] ? __pfx_kthread+0x10/0x10 <4> [298.743370] ret_from_fork+0x3d4/0x480 <4> [298.743373] ? __pfx_kthread+0x10/0x10 <4> [298.743377] ret_from_fork_asm+0x1a/0x30 <4> [298.743384] </TASK> <4> [298.743386] irq event stamp: 1932653 <4> [298.743387] hardirqs last enabled at (1932659): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.743391] hardirqs last disabled at (1932664): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.743393] softirqs last enabled at (1931250): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70 <4> [298.743396] softirqs last disabled at (1931248): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120 <4> [298.743398] ---[ end trace 0000000000000000 ]--- <6> [298.743400] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.743505] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.743522] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.743789] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.743885] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.744023] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.744247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.744347] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.744455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.744550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.744642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.744733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.744824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.744917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.745004] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.745105] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.745229] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.746243] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.756410] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.756665] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.757689] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.757769] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.757848] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.757928] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.758005] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.758081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.758156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.758232] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.758307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.758382] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.758471] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.758545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.758620] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.758694] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.758768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.758842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.758916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.758989] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.759063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.759144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.759226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.759307] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.759389] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.759480] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.759557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.759634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.759709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.759787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.759870] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.759951] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.760034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.760113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.760199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.760284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.760369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.760458] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.760538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.760616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.760692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.760769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.760851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.760933] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.761012] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.761090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.761171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.761248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.761326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.761406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.761484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.761558] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.761633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.761714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.761795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.761877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.761954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.762033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.762109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.762184] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.762264] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.762375] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.762380] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41761, lrc_seqno=41761, guc_id=0, flags=0x73 in no process [-1] <7> [298.762382] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.763303] ------------[ cut here ]------------ <4> [298.763305] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.763306] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.763378] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.763448] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.763457] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.763460] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.763461] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.763462] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.763468] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.763537] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.763539] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.763541] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.763543] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.763544] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.763545] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.763546] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.763548] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.763550] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.763551] CR2: 000072fbb0ae6000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.763553] PKRU: 55555554 <4> [298.763554] Call Trace: <4> [298.763555] <TASK> <4> [298.763559] ? lock_sync+0x100/0x100 <4> [298.763564] ? lock_release+0xd0/0x2b0 <4> [298.763570] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.763575] process_one_work+0x239/0x760 <4> [298.763582] worker_thread+0x200/0x3f0 <4> [298.763585] ? __pfx_worker_thread+0x10/0x10 <4> [298.763587] kthread+0x10d/0x150 <4> [298.763590] ? __pfx_kthread+0x10/0x10 <4> [298.763594] ret_from_fork+0x3d4/0x480 <4> [298.763596] ? __pfx_kthread+0x10/0x10 <4> [298.763599] ret_from_fork_asm+0x1a/0x30 <4> [298.763606] </TASK> <4> [298.763607] irq event stamp: 1935787 <4> [298.763609] hardirqs last enabled at (1935793): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.763612] hardirqs last disabled at (1935798): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.763614] softirqs last enabled at (1934654): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.763616] softirqs last disabled at (1934647): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.763618] ---[ end trace 0000000000000000 ]--- <6> [298.763620] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.763692] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.763698] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.764109] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.764212] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.764318] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.764524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.764613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.764706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.764795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.764882] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.764969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.765056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.765146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.765232] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.765329] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.765450] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.766456] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.776411] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.776660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.777827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.777907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.777983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.778060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.778138] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.778214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.778293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.778372] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.778461] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.778538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.778614] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.778688] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.778763] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.778838] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.778912] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.778986] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.779059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.779133] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.779207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.779288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.779371] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.779463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.779545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.779623] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.779701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.779780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.779860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.779943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.780026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.780107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.780188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.780269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.780354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.780445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.780531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.780617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.780701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.780781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.780860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.780939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.781019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.781098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.781180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.781261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.781344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.781427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.781505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.781583] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.781663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.781743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.781824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.781906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.781985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.782064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.782141] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.782222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.782299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.782375] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.782461] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.782572] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.782576] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41761, lrc_seqno=41761, guc_id=0, flags=0x73 in no process [-1] <7> [298.782578] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.782638] ------------[ cut here ]------------ <4> [298.782639] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.782640] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.782712] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.782777] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.782785] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.782788] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.782789] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.782791] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.782795] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.782866] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.782867] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.782870] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.782871] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.782872] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.782873] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.782875] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.782876] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.782878] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.782879] CR2: 000072fbb0ae6000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.782880] PKRU: 55555554 <4> [298.782882] Call Trace: <4> [298.782883] <TASK> <4> [298.782886] ? lock_sync+0x100/0x100 <4> [298.782891] ? lock_release+0xd0/0x2b0 <4> [298.782896] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.782902] process_one_work+0x239/0x760 <4> [298.782908] worker_thread+0x200/0x3f0 <4> [298.782911] ? __pfx_worker_thread+0x10/0x10 <4> [298.782913] kthread+0x10d/0x150 <4> [298.782916] ? __pfx_kthread+0x10/0x10 <4> [298.782920] ret_from_fork+0x3d4/0x480 <4> [298.782921] ? __pfx_kthread+0x10/0x10 <4> [298.782925] ret_from_fork_asm+0x1a/0x30 <4> [298.782932] </TASK> <4> [298.782933] irq event stamp: 1938879 <4> [298.782934] hardirqs last enabled at (1938885): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.782937] hardirqs last disabled at (1938890): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.782940] softirqs last enabled at (1937740): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.782942] softirqs last disabled at (1937731): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.782944] ---[ end trace 0000000000000000 ]--- <5> [298.783310] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41816, lrc_seqno=41816, guc_id=0, flags=0x73 in no process [-1] <7> [298.783316] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.783552] ------------[ cut here ]------------ <4> [298.783554] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.783557] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#12: kworker/u64:13/2465 <4> [298.783677] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.783778] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.783791] CPU: 12 UID: 0 PID: 2465 Comm: kworker/u64:13 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.783795] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.783797] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.783800] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.783809] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.783925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.783927] RSP: 0018:ffffc900042efca0 EFLAGS: 00010246 <4> [298.783931] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.783933] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.783935] RBP: ffffc900042efdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.783937] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.783939] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.783941] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [298.783943] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.783945] CR2: 000072fbb0ae2000 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [298.783947] PKRU: 55555554 <4> [298.783949] Call Trace: <4> [298.783951] <TASK> <4> [298.783957] ? lock_sync+0x100/0x100 <4> [298.783965] ? lock_release+0xd0/0x2b0 <4> [298.783974] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.783983] process_one_work+0x239/0x760 <4> [298.783994] worker_thread+0x200/0x3f0 <4> [298.783998] ? __pfx_worker_thread+0x10/0x10 <4> [298.784002] kthread+0x10d/0x150 <4> [298.784006] ? __pfx_kthread+0x10/0x10 <4> [298.784011] ret_from_fork+0x3d4/0x480 <4> [298.784014] ? __pfx_kthread+0x10/0x10 <4> [298.784019] ret_from_fork_asm+0x1a/0x30 <4> [298.784031] </TASK> <4> [298.784033] irq event stamp: 879399 <4> [298.784035] hardirqs last enabled at (879405): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.784039] hardirqs last disabled at (879410): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.784042] softirqs last enabled at (878608): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.784046] softirqs last disabled at (878601): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.784049] ---[ end trace 0000000000000000 ]--- <6> [298.784052] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.784166] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.784175] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.784257] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [298.784363] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [298.784479] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.784682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.784771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.784865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.784955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [298.785044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [298.785132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [298.785220] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [298.785308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [298.785392] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [298.785500] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [298.785617] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [298.786623] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [298.796409] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [298.796660] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [298.797663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [298.797743] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [298.797821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [298.797897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [298.797975] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [298.798053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [298.798129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [298.798205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [298.798281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [298.798357] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [298.798442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [298.798520] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [298.798597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [298.798673] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [298.798747] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [298.798821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [298.798896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [298.798970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [298.799044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [298.799126] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [298.799208] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [298.799288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [298.799369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [298.799459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [298.799537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [298.799617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [298.799700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [298.799785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [298.799868] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [298.799949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [298.800028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [298.800109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [298.800194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [298.800279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [298.800364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [298.800455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [298.800535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [298.800613] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [298.800690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [298.800771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [298.800855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [298.800937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [298.801017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [298.801096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [298.801178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [298.801256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [298.801335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [298.801415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [298.801493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [298.801570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [298.801646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [298.801726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [298.801803] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [298.801881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [298.801957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [298.802034] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [298.802114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [298.802194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [298.802277] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [298.802388] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [298.802392] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=41816, lrc_seqno=41816, guc_id=0, flags=0x73 in no process [-1] <7> [298.802394] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [298.802474] ------------[ cut here ]------------ <4> [298.802475] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out <4> [298.802477] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1626 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#2: kworker/u64:3/193 <4> [298.802548] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad <4> [298.802613] dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci] <4> [298.802621] CPU: 2 UID: 0 PID: 193 Comm: kworker/u64:3 Tainted: G S U W 7.0.0-lgci-xe-xe-4924-84de0c4efa971d100-debug+ #1 PREEMPT(lazy) <4> [298.802623] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN <4> [298.802625] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [298.802626] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [298.802631] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe] <4> [298.802703] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49 <4> [298.802704] RSP: 0018:ffffc900015c7ca0 EFLAGS: 00010246 <4> [298.802706] RAX: ffffffffa1204833 RBX: 0000000000000000 RCX: 0000000000000000 <4> [298.802708] RDX: ffff88810396d490 RSI: ffffffffa1204833 RDI: ffffffffa1003f20 <4> [298.802709] RBP: ffffc900015c7db0 R08: 0000000000000000 R09: 0000000000000000 <4> [298.802710] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [298.802711] R13: ffff88810396d490 R14: ffff88815616e818 R15: 00000000ffffffc2 <4> [298.802713] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [298.802714] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [298.802716] CR2: 000072fbb0ae6000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [298.802717] PKRU: 55555554 <4> [298.802718] Call Trace: <4> [298.802719] <TASK> <4> [298.802723] ? lock_sync+0x100/0x100 <4> [298.802728] ? lock_release+0xd0/0x2b0 <4> [298.802733] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [298.802739] process_one_work+0x239/0x760 <4> [298.802745] worker_thread+0x200/0x3f0 <4> [298.802748] ? __pfx_worker_thread+0x10/0x10 <4> [298.802750] kthread+0x10d/0x150 <4> [298.802753] ? __pfx_kthread+0x10/0x10 <4> [298.802756] ret_from_fork+0x3d4/0x480 <4> [298.802758] ? __pfx_kthread+0x10/0x10 <4> [298.802762] ret_from_fork_asm+0x1a/0x30 <4> [298.802769] </TASK> <4> [298.802770] irq event stamp: 1943115 <4> [298.802771] hardirqs last enabled at (1943121): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [298.802774] hardirqs last disabled at (1943126): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [298.802776] softirqs last enabled at (1942082): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.802779] softirqs last disabled at (1942075): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [298.802781] ---[ end trace 0000000000000000 ]--- <6> [298.802783] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [298.802852] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [298.802858] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [298.803268] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [298.803474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [298.803563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [298.803657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [298.803745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440

<4> [297.362858] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.363017] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.384391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.384592] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.406655] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.406889] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.409982] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.410211] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.432120] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.432274] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.451891] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.452042] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.453590] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr

<4> [297.453612] intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci]

<4> [297.453748] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.473660] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.473812] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.492705] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.492860] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.493426] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.493579] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.514391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.514551] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.533711] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.533863] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.535375] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.535533] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.555402] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.555561] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.574762] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.574918] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.576398] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.576573] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.595620] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.595774] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.617415] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.617574] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.618354] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.618510] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.637570] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.637722] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.659423] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.659598] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.660428] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.660600] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.679647] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.679798] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.699388] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.699576] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.701285] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.701440] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.720827] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.720980] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.741677] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.741828] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.743397] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.743550] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.762591] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.762745] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.782283] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.782439] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.784205] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.784359] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.803600] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.803753] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.823197] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.823349] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.824955] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.825189] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.845391] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.845549] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.864646] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.864799] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.865769] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.865940] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.884514] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.884667] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.904104] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.904254] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.906196] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic

<4> [297.906222] cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci]

<4> [297.906347] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.927516] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.927674] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.947648] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.947819] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.948605] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.948778] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.968832] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.968982] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.992894] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.993125] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [297.993951] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [297.994175] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.016041] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.016201] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.036991] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.037149] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.038808] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.038964] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.057839] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.057994] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.078618] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.078773] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.080362] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.080517] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.100397] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.100553] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.119651] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.119805] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.121882] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.122035] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.141346] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.141502] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.161743] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.161898] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.163179] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.163382] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.182991] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.183143] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.202874] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.203028] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.204609] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.204758] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.223406] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.223563] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.243696] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.243836] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.244743] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.244917] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.265016] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.265176] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.286042] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.286194] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.287773] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.287922] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.308357] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.308521] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.327597] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.327748] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.329182] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.329331] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.348822] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.348981] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.369558] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.369709] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.371280] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.371437] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.391408] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.391567] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.410840] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.410997] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.412580] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.412732] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.432761] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.432914] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.452710] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.452865] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.454325] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.454506] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.473492] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.473643] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.494186] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.494337] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.495919] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.496069] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.515483] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.515634] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.535728] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.535882] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.537311] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.537544] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.556795] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.556950] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.577623] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.577775] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.579212] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.579362] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.598467] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.598620] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.619457] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.619612] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.621082] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.621231] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.641395] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.641553] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.660635] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.660788] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.661604] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.661850] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.680466] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.680619] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.700065] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.700217] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.701943] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers

<4> [298.701962] i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink autofs4 [last unloaded: xe_vfio_pci]

<4> [298.702104] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.722717] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.722871] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.742245] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.742389] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.743135] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.743311] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.763378] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.763537] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.782712] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.782866] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.783677] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.783925] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

<4> [298.802548] Modules linked in: vfio_pci_core vfio_iommu_type1 vfio iommufd snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit overlay intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal hid_generic cmdlinepart binfmt_misc intel_powerclamp spi_nor eeepc_wmi mtd asus_wmi mei_pxp coretemp mei_hdcp sparse_keymap platform_profile wmi_bmof kvm_intel kvm irqbypass snd_intel_dspcfg r8169 ghash_clmulni_intel aesni_intel snd_hda_codec rapl usbhid snd_hda_core intel_cstate hid video snd_hwdep realtek snd_pcm snd_timer nls_iso8859_1 i2c_i801 i2c_mux idma64 spi_intel_pci snd spi_intel soundcore i2c_smbus intel_pmc_core pmt_telemetry pmt_discovery mei_me pmt_class mei intel_pmc_ssram_telemetry pinctrl_alderlake wmi acpi_tad intel_vsec acpi_pad

<4> [298.802703] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 86 da 5e e1 48 89 c6 48 8d 3d ac 90 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49

Results for igt@xe_pat@pt-caching-update-pat-and-pte