Panic#1 Part20
<6>[ 153.029389] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16498: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!5QCc`s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8UFFs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
Panic#1 Part19
<6>[ 153.029393] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16499: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s3L`Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!5QCc`s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s7?9js8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
Panic#1 Part18
<6>[ 153.029398] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16500: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W,Fs8W-!s8W-!s8W-!s8W-!r;Zfss8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s3L`Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s3L`Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W,Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
Panic#1 Part17
<6>[ 153.029402] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16501: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!r;Zfss8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8;oss8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!oDejjs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s3L`Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
Panic#1 Part16
<6>[ 153.029407] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16502: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W,Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8V]js8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W,js8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
Panic#1 Part15
<6>[ 153.029410] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16503: s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!ci=%Fs8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!s8W-!
<6>[ 153.029413] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16504: **** GuC CT ****
<6>[ 153.029415] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16505: CT disabled
<6>[ 153.029416] xe 0000:03:00.0: [drm] Tile0: GT0: Capture 1.16506: Done.
<7>[ 153.034894] xe 0000:03:00.0: [drm:intel_power_well_enable [xe]] enabling AUX_TC2
<7>[ 153.036040] xe 0000:03:00.0: [drm:intel_dpll_disable [xe]] disable TC PLL 2 (active 0x2, on? 1) for [CRTC:268:pipe B]
<7>[ 153.037639] xe 0000:03:00.0: [drm:intel_dpll_disable [xe]] disabling TC PLL 2
<7>[ 153.037803] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:506:DDI TC1/PHY F]
<7>[ 153.037882] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:508:DP-MST A]
<7>[ 153.037958] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:509:DP-MST B]
<7>[ 153.038033] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:510:DP-MST C]
<7>[ 153.038103] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:511:DP-MST D]
<7>[ 153.038173] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:525:DDI TC2/PHY G]
Panic#1 Part14
<7>[ 153.038246] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:527:DP-MST A]
<7>[ 153.038315] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:528:DP-MST B]
<7>[ 153.038384] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:529:DP-MST C]
<7>[ 153.038452] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:530:DP-MST D]
<7>[ 153.038519] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:537:DDI TC3/PHY H]
<7>[ 153.038585] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:541:DDI TC4/PHY I]
<7>[ 153.038657] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:543:DP-MST A]
<7>[ 153.038729] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:544:DP-MST B]
<7>[ 153.038796] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:545:DP-MST C]
<7>[ 153.038864] xe 0000:03:00.0: [drm:intel_modeset_verify_disabled [xe]] [ENCODER:546:DP-MST D]
<7>[ 153.038931] xe 0000:03:00.0: [drm:verify_connector_state [xe]] [CONNECTOR:526:DP-2]
<7>[ 153.039020] xe 0000:03:00.0: [drm:intel_dbuf_mdclk_cdclk_ratio_update [xe]] Updating dbuf ratio to 2 (mbus joined: no)
<7>[ 153.039095] xe 0000:03:00.0: [drm:intel_dbuf_mbus_join_update [xe]] Changing mbus joined: yes -> no (pipe: *)
<7>[ 153.039164] xe 0000:03:00.0: [drm:gen9_dbuf_slices_update [xe]] Updating dbuf slices to 0x1
<7>[ 153.039336] xe 0000:03:00.0: [drm:intel_modeset_verify_crtc [xe]] [CRTC:268:pipe B]
<7>[ 153.039477] xe 0000:03:00.0: [drm:intel_pmdemand_program_params [xe]] initiate pmdemand request values: (0x1e03010 0x28d00071)
Panic#1 Part13
<7>[ 153.040385] xe 0000:03:00.0: [drm:xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608
<5>[ 153.040690] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=7774, lrc_seqno=7774, guc_id=0, flags=0x73 in no process [-1]
<7>[ 153.040696] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4>[ 153.040764] ------------[ cut here ]------------
<4>[ 153.040766] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4>[ 153.040767] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:53/3837
<4>[ 153.040838] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp cmdlinepart hid_generic spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile kvm_intel wmi_bmof kvm irqbypass ghash_clmulni_intel usbhid aesni_intel hid rapl snd_intel_dspcfg snd_hda_codec intel_cstate r8169 snd_hda_core snd_hwdep binfmt_misc snd_pcm realtek video snd_timer snd idma64 i2c_i801 i2c_mux mei_me spi_intel_pci spi_intel i2c_smbus soundcore mei intel_pmc_core pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
Panic#1 Part12
<4>[ 153.040897] autofs4 [last unloaded: snd_hda_intel]
<4>[ 153.040901] CPU: 5 UID: 0 PID: 3837 Comm: kworker/u64:53 Tainted: G S U W L 7.0.0-rc3-lgci-xe-xe-pw-163019v1-debug+ #1 PREEMPT(lazy)
<4>[ 153.040904] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4>[ 153.040905] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4>[ 153.040906] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4>[ 153.040912] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4>[ 153.040979] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 58 5e e1 48 89 c6 48 8d 3d dc a3 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4>[ 153.040980] RSP: 0018:ffffc9000a1f7ca0 EFLAGS: 00010246
<4>[ 153.040982] RAX: ffffffffa11fd811 RBX: 0000000000000000 RCX: 0000000000000000
<4>[ 153.040984] RDX: ffff888104b9f190 RSI: ffffffffa11fd811 RDI: ffffffffa1003da0
Panic#1 Part11
<4>[ 153.040985] RBP: ffffc9000a1f7db0 R08: 0000000000000000 R09: 0000000000000000
<4>[ 153.040986] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4>[ 153.040987] R13: ffff888104b9f190 R14: ffff888186415018 R15: 00000000ffffffc2
<4>[ 153.040988] FS: 0000000000000000(0000) GS:ffff8888daf1b000(0000) knlGS:0000000000000000
<4>[ 153.040989] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[ 153.040990] CR2: 0000762ea34056f0 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4>[ 153.040991] PKRU: 55555554
<4>[ 153.040992] Call Trace:
<4>[ 153.040993]
<4>[ 153.040998] ? lock_acquire+0x40/0x2f0
<4>[ 153.041004] ? lock_release+0xd0/0x2b0
<4>[ 153.041008] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4>[ 153.041013] process_one_work+0x22e/0x740
<4>[ 153.041019] worker_thread+0x1e8/0x3d0
<4>[ 153.041021] ? __pfx_worker_thread+0x10/0x10
<4>[ 153.041023] kthread+0x10d/0x150
<4>[ 153.041026] ? __pfx_kthread+0x10/0x10
<4>[ 153.041029] ret_from_fork+0x3d4/0x480
<4>[ 153.041031] ? __pfx_kthread+0x10/0x10
<4>[ 153.041034] ret_from_fork_asm+0x1a/0x30
<4>[ 153.041041]
<4>[ 153.041042] irq event stamp: 115183
<4>[ 153.041043] hardirqs last enabled at (115189): [] __up_console_sem+0x79/0xa0
<4>[ 153.041046] hardirqs last disabled at (115194): [] __up_console_sem+0x5e/0xa0
<4>[ 153.041048] softirqs last enabled at (114384): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041050] softirqs last disabled at (114377): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041052] ---[ end trace 0000000000000000 ]---
Panic#1 Part10
<6>[ 153.041054] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<5>[ 153.041125] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=7774, lrc_seqno=7774, guc_id=0, flags=0x73 in no process [-1]
<7>[ 153.041127] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4>[ 153.041185] ------------[ cut here ]------------
<4>[ 153.041185] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4>[ 153.041187] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:53/3837
<4>[ 153.041254] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp cmdlinepart hid_generic spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile kvm_intel wmi_bmof kvm irqbypass ghash_clmulni_intel usbhid aesni_intel hid rapl snd_intel_dspcfg snd_hda_codec intel_cstate r8169 snd_hda_core snd_hwdep binfmt_misc snd_pcm realtek video snd_timer snd idma64 i2c_i801 i2c_mux mei_me spi_intel_pci spi_intel i2c_smbus soundcore mei intel_pmc_core pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
Panic#1 Part9
<4>[ 153.041308] autofs4 [last unloaded: snd_hda_intel]
<4>[ 153.041311] CPU: 5 UID: 0 PID: 3837 Comm: kworker/u64:53 Tainted: G S U W L 7.0.0-rc3-lgci-xe-xe-pw-163019v1-debug+ #1 PREEMPT(lazy)
<4>[ 153.041314] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4>[ 153.041314] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4>[ 153.041315] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4>[ 153.041320] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4>[ 153.041380] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 58 5e e1 48 89 c6 48 8d 3d dc a3 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4>[ 153.041381] RSP: 0018:ffffc9000a1f7ca0 EFLAGS: 00010246
<4>[ 153.041383] RAX: ffffffffa11fd811 RBX: 0000000000000000 RCX: 0000000000000000
<4>[ 153.041384] RDX: ffff888104b9f190 RSI: ffffffffa11fd811 RDI: ffffffffa1003da0
Panic#1 Part8
<4>[ 153.041385] RBP: ffffc9000a1f7db0 R08: 0000000000000000 R09: 0000000000000000
<4>[ 153.041386] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
<4>[ 153.041387] R13: ffff888104b9f190 R14: ffff888186415018 R15: 00000000ffffffc2
<4>[ 153.041388] FS: 0000000000000000(0000) GS:ffff8888daf1b000(0000) knlGS:0000000000000000
<4>[ 153.041389] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[ 153.041390] CR2: 0000762ea34056f0 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4>[ 153.041391] PKRU: 55555554
<4>[ 153.041392] Call Trace:
<4>[ 153.041393]
<4>[ 153.041396] ? lock_acquire+0x40/0x2f0
<4>[ 153.041401] ? lock_release+0xd0/0x2b0
<4>[ 153.041405] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4>[ 153.041410] process_one_work+0x22e/0x740
<4>[ 153.041415] worker_thread+0x1e8/0x3d0
<4>[ 153.041417] ? __pfx_worker_thread+0x10/0x10
<4>[ 153.041419] kthread+0x10d/0x150
<4>[ 153.041422] ? __pfx_kthread+0x10/0x10
<4>[ 153.041425] ret_from_fork+0x3d4/0x480
<4>[ 153.041426] ? __pfx_kthread+0x10/0x10
<4>[ 153.041429] ret_from_fork_asm+0x1a/0x30
<4>[ 153.041435]
<4>[ 153.041436] irq event stamp: 116027
<4>[ 153.041437] hardirqs last enabled at (116033): [] __up_console_sem+0x79/0xa0
<4>[ 153.041439] hardirqs last disabled at (116038): [] __up_console_sem+0x5e/0xa0
<4>[ 153.041440] softirqs last enabled at (114384): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041442] softirqs last disabled at (114377): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041444] ---[ end trace 0000000000000000 ]---
Panic#1 Part7
<6>[ 153.041445] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe]
<5>[ 153.041510] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=7774, lrc_seqno=7774, guc_id=0, flags=0x73 in no process [-1]
<7>[ 153.041512] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken
<4>[ 153.041570] ------------[ cut here ]------------
<4>[ 153.041571] xe 0000:03:00.0: [drm] Tile0: GT0: Kernel-submitted job timed out
<4>[ 153.041572] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1612 at guc_exec_queue_timedout_job+0x1424/0x2400 [xe], CPU#5: kworker/u64:53/3837
<4>[ 153.041645] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_lb mei_gsc_proxy mei_gsc mtd_intel_dg xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling x86_pkg_temp_thermal intel_powerclamp coretemp cmdlinepart hid_generic spi_nor eeepc_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile kvm_intel wmi_bmof kvm irqbypass ghash_clmulni_intel usbhid aesni_intel hid rapl snd_intel_dspcfg snd_hda_codec intel_cstate r8169 snd_hda_core snd_hwdep binfmt_misc snd_pcm realtek video snd_timer snd idma64 i2c_i801 i2c_mux mei_me spi_intel_pci spi_intel i2c_smbus soundcore mei intel_pmc_core pmt_telemetry nls_iso8859_1 pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec acpi_pad acpi_tad pinctrl_alderlake dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink
Panic#1 Part6
<4>[ 153.041696] autofs4 [last unloaded: snd_hda_intel]
<4>[ 153.041699] CPU: 5 UID: 0 PID: 3837 Comm: kworker/u64:53 Tainted: G S U W L 7.0.0-rc3-lgci-xe-xe-pw-163019v1-debug+ #1 PREEMPT(lazy)
<4>[ 153.041701] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4>[ 153.041702] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4>[ 153.041703] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched]
<4>[ 153.041706] RIP: 0010:guc_exec_queue_timedout_job+0x142d/0x2400 [xe]
<4>[ 153.041765] Code: 74 04 48 8b 7f 08 4c 8b 6f 50 4d 85 ed 75 03 4c 8b 2f e8 66 58 5e e1 48 89 c6 48 8d 3d dc a3 39 00 41 89 d8 44 89 e1 4c 89 ea <67> 48 0f b9 3a 48 8b 45 90 48 8b 40 60 e9 c6 ee ff ff 8b 70 08 49
<4>[ 153.041767] RSP: 0018:ffffc9000a1f7ca0 EFLAGS: 00010246
<4>[ 153.041768] RAX: ffffffffa11fd811 RBX: 0000000000000000 RCX: 0000000000000000
<4>[ 153.041769] RDX: ffff888104b9f190 RSI: ffffffffa11fd811 RDI: ffffffffa1003da0
<4>[ 153.041770] RBP: ffffc9000a1f7db0 R08: 0000000000000000 R09: 0000000000000000
<4>[ 153.041771] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
Panic#1 Part5
<4>[ 153.041772] R13: ffff888104b9f190 R14: ffff888186415018 R15: 00000000ffffffc2
<4>[ 153.041773] FS: 0000000000000000(0000) GS:ffff8888daf1b000(0000) knlGS:0000000000000000
<4>[ 153.041774] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
<4>[ 153.041775] CR2: 0000762ea34056f0 CR3: 000000000344c003 CR4: 0000000000f72ef0
<4>[ 153.041776] PKRU: 55555554
<4>[ 153.041777] Call Trace:
<4>[ 153.041778]
<4>[ 153.041781] ? lock_acquire+0x40/0x2f0
<4>[ 153.041785] ? lock_release+0xd0/0x2b0
<4>[ 153.041790] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched]
<4>[ 153.041794] process_one_work+0x22e/0x740
<4>[ 153.041799] worker_thread+0x1e8/0x3d0
<4>[ 153.041801] ? __pfx_worker_thread+0x10/0x10
<4>[ 153.041803] kthread+0x10d/0x150
<4>[ 153.041806] ? __pfx_kthread+0x10/0x10
<4>[ 153.041808] ret_from_fork+0x3d4/0x480
<4>[ 153.041810] ? __pfx_kthread+0x10/0x10
<4>[ 153.041813] ret_from_fork_asm+0x1a/0x30
<4>[ 153.041819]
<4>[ 153.041820] irq event stamp: 116879
<4>[ 153.041820] hardirqs last enabled at (116885): [] __up_console_sem+0x79/0xa0
<4>[ 153.041822] hardirqs last disabled at (116890): [] __up_console_sem+0x5e/0xa0
<4>[ 153.041824] softirqs last enabled at (116136): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041826] softirqs last disabled at (116131): [] __irq_exit_rcu+0x13f/0x160
<4>[ 153.041828] ---[ end trace 0000000000000000 ]---
<7>[ 153.143677] xe 0000:03:00.0: [drm:intel_power_well_disable [xe]] disabling AUX_TC2
<7>[ 153.224042] xe 0000:03:00.0: [drm:xe_gt_suspend [xe]] Tile0: GT0: suspending
Panic#1 Part4
<0>[ 214.454637] xe 0000:03:00.0: PM: **** DPM device timeout ****
<0>[ 214.454678] Call Trace:
<0>[ 214.454680]
<0>[ 214.454683] dpm_watchdog_handler+0x9d/0xf0
<0>[ 214.454688] ? __pfx_dpm_watchdog_handler+0x10/0x10
<0>[ 214.454691] call_timer_fn+0xa8/0x290
<0>[ 214.454696] ? __pfx_dpm_watchdog_handler+0x10/0x10
<0>[ 214.454699] __run_timers+0x226/0x310
<0>[ 214.454702] ? mark_held_locks+0x46/0x90
<0>[ 214.454707] timer_expire_remote+0x46/0x70
<0>[ 214.454710] tmigr_handle_remote+0x491/0x5a0
<0>[ 214.454715] ? run_timer_softirq+0x7e/0xe0
<0>[ 214.454718] ? _raw_spin_unlock_irq+0x27/0x70
<0>[ 214.454721] ? run_timer_softirq+0x7e/0xe0
<0>[ 214.454723] ? trace_hardirqs_on+0x22/0x100
<0>[ 214.454728] run_timer_softirq+0xcf/0xe0
<0>[ 214.454730] handle_softirqs+0xd2/0x500
<0>[ 214.454735] __irq_exit_rcu+0x13f/0x160
<0>[ 214.454737] irq_exit_rcu+0xe/0x20
<0>[ 214.454740] sysvec_apic_timer_interrupt+0xa0/0xc0
<0>[ 214.454743]
<0>[ 214.454744]
<0>[ 214.454746] asm_sysvec_apic_timer_interrupt+0x1b/0x20
<0>[ 214.454749] RIP: 0010:lock_is_held_type+0xe8/0x130
<0>[ 214.454752] Code: c0 44 39 f0 41 0f 94 c0 b8 ff ff ff ff 65 0f c1 05 35 65 65 01 83 f8 01 75 49 48 f7 45 d0 00 02 00 00 74 06 fb 0f 1f 44 00 00 <48> 83 c4 08 44 89 c0 5b 41 5c 41 5d 41 5e 41 5f 5d 31 d2 31 c9 31
<0>[ 214.454754] RSP: 0018:ffffc9000a1ffb08 EFLAGS: 00000206
<0>[ 214.454757] RAX: 0000000000000001 RBX: ffff8881b8f54540 RCX: 0000000000000000
<0>[ 214.454759] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
<0>[ 214.454760] RBP: ffffc9000a1ffb38 R08: 0000000000000000 R09: 0000000000000000
Panic#1 Part3
<0>[ 214.454761] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff835c5ba0
<0>[ 214.454763] R13: ffff8881b8f53540 R14: 00000000ffffffff R15: 0000000000000004
<0>[ 214.454768] ? lock_is_held_type+0xa3/0x130
<0>[ 214.454772] __might_resched+0x254/0x2d0
<0>[ 214.454776] __might_sleep+0x49/0x60
<0>[ 214.454779] xe_guc_submit_reset_wait+0x32/0x100 [xe]
<0>[ 214.454868] xe_guc_reset_wait+0xe/0x20 [xe]
<0>[ 214.454926] xe_uc_suspend+0x44/0x90 [xe]
<0>[ 214.455004] xe_gt_suspend+0x75/0x140 [xe]
<0>[ 214.455062] xe_pm_suspend+0x206/0x2e0 [xe]
<0>[ 214.455130] xe_pci_suspend+0x27/0x80 [xe]
<0>[ 214.455195] pci_pm_suspend+0x7f/0x190
<0>[ 214.455199] ? __pfx_pci_pm_suspend+0x10/0x10
<0>[ 214.455202] dpm_run_callback+0x6d/0x270
<0>[ 214.455206] device_suspend+0x22d/0x7c0
<0>[ 214.455210] ? __pfx_dpm_watchdog_handler+0x10/0x10
<0>[ 214.455215] async_suspend+0x1d/0x40
<0>[ 214.455218] async_run_entry_fn+0x35/0x170
<0>[ 214.455221] process_one_work+0x22e/0x740
<0>[ 214.455227] worker_thread+0x1e8/0x3d0
<0>[ 214.455229] ? __pfx_worker_thread+0x10/0x10
<0>[ 214.455231] kthread+0x10d/0x150
<0>[ 214.455234] ? __pfx_kthread+0x10/0x10
<0>[ 214.455238] ret_from_fork+0x3d4/0x480
<0>[ 214.455240] ? __pfx_kthread+0x10/0x10
<0>[ 214.455243] ret_from_fork_asm+0x1a/0x30
<0>[ 214.455250]
<0>[ 214.455252] Kernel panic - not syncing: xe 0000:03:00.0: unrecoverable failure
<0>[ 214.455426] Kernel Offset: disabled
<4>[ 214.455440] CPU: 5 UID: 0 PID: 3838 Comm: kworker/u64:54 Tainted: G S U W L 7.0.0-rc3-lgci-xe-xe-pw-163019v1-debug+ #1 PREEMPT(lazy)
Panic#1 Part2
<4>[ 214.455443] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP
<4>[ 214.455445] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024
<4>[ 214.455446] Workqueue: async async_run_entry_fn
<4>[ 214.455450] Call Trace:
<4>[ 214.455451]
<4>[ 214.455453] dump_stack_lvl+0x25/0xf0
<4>[ 214.455456] dump_stack+0x10/0x20
<4>[ 214.455458] vpanic+0x4a2/0x540
<4>[ 214.455461] panic+0x57/0x60
<4>[ 214.455465] dpm_watchdog_handler+0xdc/0xf0
<4>[ 214.455468] ? __pfx_dpm_watchdog_handler+0x10/0x10
<4>[ 214.455471] call_timer_fn+0xa8/0x290
<4>[ 214.455475] ? __pfx_dpm_watchdog_handler+0x10/0x10
<4>[ 214.455477] __run_timers+0x226/0x310
<4>[ 214.455480] ? mark_held_locks+0x46/0x90
<4>[ 214.455485] timer_expire_remote+0x46/0x70
<4>[ 214.455488] tmigr_handle_remote+0x491/0x5a0
<4>[ 214.455493] ? run_timer_softirq+0x7e/0xe0
<4>[ 214.455495] ? _raw_spin_unlock_irq+0x27/0x70
<4>[ 214.455498] ? run_timer_softirq+0x7e/0xe0
<4>[ 214.455500] ? trace_hardirqs_on+0x22/0x100
<4>[ 214.455504] run_timer_softirq+0xcf/0xe0
<4>[ 214.455506] handle_softirqs+0xd2/0x500
<4>[ 214.455511] __irq_exit_rcu+0x13f/0x160
<4>[ 214.455513] irq_exit_rcu+0xe/0x20
<4>[ 214.455516] sysvec_apic_timer_interrupt+0xa0/0xc0
<4>[ 214.455518]
<4>[ 214.455519]
<4>[ 214.455521] asm_sysvec_apic_timer_interrupt+0x1b/0x20
<4>[ 214.455523] RIP: 0010:lock_is_held_type+0xe8/0x130
<4>[ 214.455526] Code: c0 44 39 f0 41 0f 94 c0 b8 ff ff ff ff 65 0f c1 05 35 65 65 01 83 f8 01 75 49 48 f7 45 d0 00 02 00 00 74 06 fb 0f 1f 44 00 00 <48> 83 c4 08 44 89 c0 5b 41 5c 41 5d 41 5e 41 5f 5d 31 d2 31 c9 31
Panic#1 Part1
<4>[ 214.455528] RSP: 0018:ffffc9000a1ffb08 EFLAGS: 00000206
<4>[ 214.455530] RAX: 0000000000000001 RBX: ffff8881b8f54540 RCX: 0000000000000000
<4>[ 214.455531] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 0000000000000000
<4>[ 214.455533] RBP: ffffc9000a1ffb38 R08: 0000000000000000 R09: 0000000000000000
<4>[ 214.455534] R10: 0000000000000000 R11: 0000000000000000 R12: ffffffff835c5ba0
<4>[ 214.455535] R13: ffff8881b8f53540 R14: 00000000ffffffff R15: 0000000000000004
<4>[ 214.455540] ? lock_is_held_type+0xa3/0x130
<4>[ 214.455544] __might_resched+0x254/0x2d0
<4>[ 214.455548] __might_sleep+0x49/0x60
<4>[ 214.455550] xe_guc_submit_reset_wait+0x32/0x100 [xe]
<4>[ 214.455622] xe_guc_reset_wait+0xe/0x20 [xe]
<4>[ 214.455685] xe_uc_suspend+0x44/0x90 [xe]
<4>[ 214.455769] xe_gt_suspend+0x75/0x140 [xe]
<4>[ 214.455831] xe_pm_suspend+0x206/0x2e0 [xe]
<4>[ 214.455905] xe_pci_suspend+0x27/0x80 [xe]
<4>[ 214.455976] pci_pm_suspend+0x7f/0x190
<4>[ 214.455978] ? __pfx_pci_pm_suspend+0x10/0x10
<4>[ 214.455981] dpm_run_callback+0x6d/0x270
<4>[ 214.455985] device_suspend+0x22d/0x7c0
<4>[ 214.455989] ? __pfx_dpm_watchdog_handler+0x10/0x10
<4>[ 214.455994] async_suspend+0x1d/0x40
<4>[ 214.455997] async_run_entry_fn+0x35/0x170
<4>[ 214.456000] process_one_work+0x22e/0x740
<4>[ 214.456005] worker_thread+0x1e8/0x3d0
<4>[ 214.456007] ? __pfx_worker_thread+0x10/0x10
<4>[ 214.456009] kthread+0x10d/0x150
<4>[ 214.456012] ? __pfx_kthread+0x10/0x10
<4>[ 214.456015] ret_from_fork+0x3d4/0x480
<4>[ 214.456017] ? __pfx_kthread+0x10/0x10
<4>[ 214.456020] ret_from_fork_asm+0x1a/0x30
<4>[ 214.456027]