igt@xe_evict@evict-beng-cm-threads-small on shard-bmg@xe-4916-659eae6a034b720c87d0ab101c9121efdbbbe1a5 Test details

Detail	Value
Duration	unknown
Hostname	shard-bmg-2
Igt-Version	IGT-Version: 2.4-g9b95600c4 (x86_64) (Linux: 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ x86_64)
Out	Using IGT_SRANDOM=1776375750 for randomisation Opened device: /dev/dri/card0 Starting subtest: evict-beng-cm-threads-small [thread:6589] Stack trace: [thread:6589] #0 ../lib/igt_core.c:2075 __igt_fail_assert() [thread:6589] #1 ../tests/intel/xe_evict.c:321 test_evict_cm() [thread:6589] #2 ../tests/intel/xe_evict.c:442 thread() [thread:6589] #3 [pthread_condattr_setpshared+0x684] [thread:6589] #4 [__clone+0x24c] [thread:6590] Stack trace: [thread:6590] #0 ../lib/igt_core.c:2075 __igt_fail_assert() [thread:6590] #1 ../tests/intel/xe_evict.c:321 test_evict_cm() [thread:6590] #2 ../tests/intel/xe_evict.c:442 thread() [thread:6590] #3 [pthread_condattr_setpshared+0x684] [thread:6590] #4 [__clone+0x24c] Subtest evict-beng-cm-threads-small: FAIL (76.649s) This test caused an abort condition: Kernel badly tainted (0x4244, 0x200) (check dmesg for details): TAINT_WARN: WARN_ON has happened.
Err	Starting subtest: evict-beng-cm-threads-small (xe_evict:6588) [thread:6589] CRITICAL: Test assertion failure function test_evict_cm, file ../tests/intel/xe_evict.c:378: (xe_evict:6588) [thread:6589] CRITICAL: Failed assertion: igt_ioctl(fd, (((1U) << (((0+8)+8)+14)) \| ((('d')) << (0+8)) \| (((0x40 + 0x09)) << 0) \| ((((sizeof(struct drm_xe_exec)))) << ((0+8)+8))), &exec) == 0 (xe_evict:6588) [thread:6589] CRITICAL: Last errno: 125, Operation canceled (xe_evict:6588) [thread:6589] CRITICAL: error: -1 != 0 (xe_evict:6588) [thread:6590] CRITICAL: Test assertion failure function test_evict_cm, file ../tests/intel/xe_evict.c:378: (xe_evict:6588) [thread:6590] CRITICAL: Failed assertion: igt_ioctl(fd, (((1U) << (((0+8)+8)+14)) \| ((('d')) << (0+8)) \| (((0x40 + 0x09)) << 0) \| ((((sizeof(struct drm_xe_exec)))) << ((0+8)+8))), &exec) == 0 (xe_evict:6588) [thread:6590] CRITICAL: Last errno: 125, Operation canceled (xe_evict:6588) [thread:6590] CRITICAL: error: -1 != 0 (xe_evict:6588) igt_thread-CRITICAL: Failure in a thread! Subtest evict-beng-cm-threads-small failed. ** DEBUG (xe_evict:6588) DEBUG: num_threads: 2 bo_size : 100663296 total_size : 188 (xe_evict:6588) DEBUG: Max working set 252 n_execs 128 (xe_evict:6588) DEBUG: Test requirement passed: !(!ws) (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 0: 0x0000000c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 1: 0x0000000e (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 2: 0x0000000f (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 3: 0x0000003c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 4: 0x00000032 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 5: 0x0000003e (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 6: 0x0000041c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 7: 0x00000033 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 8: 0x00000030 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 9: 0x0000020c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 10: 0x00000230 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 11: 0x0000061c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 12: 0x0000023c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 13: 0x00000000 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 14: 0x00000200 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 15: 0x00000614 (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 16: 0x00000000 (reserved) (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 17: 0x00000000 (reserved) (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 18: 0x00000000 (reserved) (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 19: 0x00000000 (reserved) (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 20: 0x0000004c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 21: 0x0000024c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 22: 0x0000004e (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 23: 0x0000004f (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 24: 0x0000008c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 25: 0x0000028c (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 26: 0x0000008e (xe_evict:6588) [thread:6590] intel_pat-DEBUG: Parsed PAT entry 27: 0x0000008f (xe_evict:6588) [thread:6589] CRITICAL: Test assertion failure function test_evict_cm, file ../tests/intel/xe_evict.c:378: (xe_evict:6588) [thread:6589] CRITICAL: Failed assertion: igt_ioctl(fd, (((1U) << (((0+8)+8)+14)) \| ((('d')) << (0+8)) \| (((0x40 + 0x09)) << 0) \| ((((sizeof(struct drm_xe_exec)))) << ((0+8)+8))), &exec) == 0 (xe_evict:6588) [thread:6589] CRITICAL: Last errno: 125, Operation canceled (xe_evict:6588) [thread:6589] CRITICAL: error: -1 != 0 (xe_evict:6588) [thread:6589] igt_core-INFO: Stack trace: (xe_evict:6588) [thread:6589] igt_core-INFO: #0 ../lib/igt_core.c:2075 __igt_fail_assert() (xe_evict:6588) [thread:6589] igt_core-INFO: #1 ../tests/intel/xe_evict.c:321 test_evict_cm() (xe_evict:6588) [thread:6589] igt_core-INFO: #2 ../tests/intel/xe_evict.c:442 thread() (xe_evict:6588) [thread:6589] igt_core-INFO: #3 [pthread_condattr_setpshared+0x684] (xe_evict:6588) [thread:6589] igt_core-INFO: #4 [__clone+0x24c] (xe_evict:6588) [thread:6590] CRITICAL: Test assertion failure function test_evict_cm, file ../tests/intel/xe_evict.c:378: (xe_evict:6588) [thread:6590] CRITICAL: Failed assertion: igt_ioctl(fd, (((1U) << (((0+8)+8)+14)) \| ((('d')) << (0+8)) \| (((0x40 + 0x09)) << 0) \| ((((sizeof(struct drm_xe_exec)))) << ((0+8)+8))), &exec) == 0 (xe_evict:6588) [thread:6590] CRITICAL: Last errno: 125, Operation canceled (xe_evict:6588) [thread:6590] CRITICAL: error: -1 != 0 (xe_evict:6588) [thread:6590] igt_core-INFO: Stack trace: (xe_evict:6588) [thread:6590] igt_core-INFO: #0 ../lib/igt_core.c:2075 __igt_fail_assert() (xe_evict:6588) [thread:6590] igt_core-INFO: #1 ../tests/intel/xe_evict.c:321 test_evict_cm() (xe_evict:6588) [thread:6590] igt_core-INFO: #2 ../tests/intel/xe_evict.c:442 thread() (xe_evict:6588) [thread:6590] igt_core-INFO: #3 [pthread_condattr_setpshared+0x684] (xe_evict:6588) [thread:6590] igt_core-INFO: #4 [__clone+0x24c] (xe_evict:6588) igt_thread-CRITICAL: Failure in a thread! END ** Subtest evict-beng-cm-threads-small: FAIL (76.649s)
Dmesg Scroll to first warning	<6> [443.284093] Console: switching to colour dummy device 80x25 <6> [443.284283] [IGT] xe_evict: executing <6> [443.301183] xe_evict (6588): drop_caches: 4 <7> [443.642356] xe 0000:03:00.0: [drm:__xe_svm_handle_pagefault [xe]] Get pages failed, falling back to retrying, asid=365, gpusvm=ffff8881ae9d9190, errno=-EFAULT <7> [443.642552] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.642696] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.642913] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.643088] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.643297] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.643490] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.643712] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.643929] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644111] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644264] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644439] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644602] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644737] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.644926] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.645061] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.645239] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.645405] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.645583] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.645747] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.646049] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.646277] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.646488] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.646699] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.646945] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.647157] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.647434] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <7> [443.648183] xe 0000:03:00.0: [drm:xe_svm_garbage_collector [xe]] Skipping madvise reset for vma. <6> [443.648455] xe 0000:03:00.0: [drm] Tile0: GT0: ASID: 365 Faulted Address: 0x00007c2c14005000 FaultType: 0 AccessType: 0 FaultLevel: 0 EngineClass: 5 ccs EngineInstance: 0 <6> [443.648551] xe 0000:03:00.0: [drm] Tile0: GT0: Fault response: Unsuccessful -EFAULT <6> [443.650181] [IGT] xe_evict: starting subtest evict-beng-cm-threads-small <7> [443.654163] xe 0000:03:00.0: [drm:drm_pagemap_dev_unhold_work [drm_gpusvm_helper]] Releasing reference on provider device and module. <6> [456.450203] pcieport 0000:00:06.0: AER: Multiple Correctable error message received from 0000:05:00.0 <4> [456.450229] nvme 0000:05:00.0: PCIe Bus Error: severity=Correctable, type=Physical Layer, (Receiver ID) <4> [456.450239] nvme 0000:05:00.0: device [15b7:5017] error status/mask=00000001/0000e000 <4> [456.450248] nvme 0000:05:00.0: [ 0] RxErr (First) <7> [456.650602] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x35353233 <7> [456.650781] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x34343435 <6> [456.756492] pcieport 0000:00:06.0: AER: Multiple Correctable error message received from 0000:05:00.0 <4> [456.756518] nvme 0000:05:00.0: PCIe Bus Error: severity=Correctable, type=Physical Layer, (Receiver ID) <4> [456.756527] nvme 0000:05:00.0: device [15b7:5017] error status/mask=00000001/0000e000 <4> [456.756537] nvme 0000:05:00.0: [ 0] RxErr (First) <7> [471.674909] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x35353232 <7> [471.675072] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x34343435 <7> [484.206184] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [484.206185] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [484.206887] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [484.206933] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [484.207003] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [484.207115] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [484.207121] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [484.207229] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [484.207315] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [484.207364] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [484.207475] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [484.207489] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [484.207590] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [484.207702] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [484.207693] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [484.207807] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [484.207868] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [484.207911] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [484.208017] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [484.208042] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [484.208218] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [484.208392] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [484.208583] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [484.208757] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [484.880592] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [484.884782] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [484.884945] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [484.885077] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [484.885207] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [484.885333] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [484.885454] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [484.885602] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [484.885721] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [484.885839] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [484.885958] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [484.886069] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [484.886260] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [484.886596] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [484.886726] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [484.886840] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [484.886948] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [484.887070] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [484.887186] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [484.887302] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [484.887419] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [484.887697] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [484.887821] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [484.887939] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [486.630805] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x35353233 <7> [486.631067] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x34353535 <4> [491.009578] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=3, not started <4> [496.128780] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=3, not started <4> [501.249949] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=3, not started <5> [501.250196] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967169, lrc_seqno=4294967169, guc_id=3, flags=0x4 in no process [-1] <7> [501.250222] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [501.250705] ------------[ cut here ]------------ <4> [501.250712] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [501.250720] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:4/203 <4> [501.251136] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [501.251527] autofs4 [last unloaded: snd_hda_intel] <4> [501.251552] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [501.251568] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [L]=SOFTLOCKUP <4> [501.251575] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [501.251584] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [501.251614] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [501.252009] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [501.252018] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [501.252031] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [501.252038] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [501.252045] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [501.252051] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [501.252057] R13: ffff888104cd0010 R14: ffff88818436a600 R15: 00000000ffffffc2 <4> [501.252064] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [501.252073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [501.252080] CR2: 000071e2538aa058 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [501.252088] PKRU: 55555554 <4> [501.252094] Call Trace: <4> [501.252101] <TASK> <4> [501.252122] ? lock_sync+0x100/0x100 <4> [501.252148] ? __pfx_autoremove_wake_function+0x10/0x10 <4> [501.252176] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [501.252206] process_one_work+0x239/0x760 <4> [501.252239] worker_thread+0x200/0x3f0 <4> [501.252253] ? __pfx_worker_thread+0x10/0x10 <4> [501.252265] kthread+0x10d/0x150 <4> [501.252281] ? __pfx_kthread+0x10/0x10 <4> [501.252300] ret_from_fork+0x3d4/0x480 <4> [501.252311] ? __pfx_kthread+0x10/0x10 <4> [501.252328] ret_from_fork_asm+0x1a/0x30 <4> [501.252367] </TASK> <4> [501.252373] irq event stamp: 654719 <4> [501.252379] hardirqs last enabled at (654725): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [501.252395] hardirqs last disabled at (654730): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [501.252406] softirqs last enabled at (653836): [<ffffffff8280edad>] addrconf_verify_rtnl+0x25d/0xa00 <4> [501.252418] softirqs last disabled at (653830): [<ffffffff8280eb83>] addrconf_verify_rtnl+0x33/0xa00 <4> [501.252427] ---[ end trace 0000000000000000 ]--- <7> [501.263085] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [501.263408] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [501.263579] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [501.263701] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [501.263823] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [501.263941] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [501.264057] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [501.264172] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [501.264291] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [501.264408] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [501.264534] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [501.264650] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [501.264791] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [501.264909] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [501.265023] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [501.265138] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [501.265252] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [501.265370] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [501.265493] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [501.265611] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [501.265729] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [501.265846] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [501.265960] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [501.266073] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [501.273213] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [501.273334] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [501.273447] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [501.273567] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [501.273688] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [501.273807] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [501.273924] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [501.274040] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [501.274159] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [501.274276] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [501.274393] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [501.274520] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [501.274758] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [501.274879] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [501.274997] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [501.275112] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [501.275228] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [501.275343] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [501.275579] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [501.275705] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [501.275825] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [501.275943] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [501.276060] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [501.276176] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [501.647282] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x35363333 <7> [501.647427] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x35353536 <4> [507.392797] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=1, not started <6> [507.505610] pcieport 0000:00:06.0: AER: Multiple Correctable error message received from 0000:05:00.0 <4> [507.505636] nvme 0000:05:00.0: PCIe Bus Error: severity=Correctable, type=Physical Layer, (Receiver ID) <4> [507.505646] nvme 0000:05:00.0: device [15b7:5017] error status/mask=00000001/0000e000 <4> [507.505656] nvme 0000:05:00.0: [ 0] RxErr (First) <4> [512.512655] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=1, not started <7> [516.637868] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 0 val 0x36363334 <7> [516.638029] xe 0000:03:00.0: [drm:xe_hwmon_read [xe]] thermal data for group 1 val 0x35353536 <4> [517.632684] xe 0000:03:00.0: [drm] Tile0: GT0: Check job timeout: seqno=4294967169, lrc_seqno=4294967169, guc_id=1, not started <5> [517.632975] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967169, lrc_seqno=4294967169, guc_id=1, flags=0x4 in no process [-1] <7> [517.633001] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [517.633486] ------------[ cut here ]------------ <4> [517.633493] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [517.633502] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#5: kworker/u64:13/3305 <4> [517.633917] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [517.634264] autofs4 [last unloaded: snd_hda_intel] <4> [517.634288] CPU: 5 UID: 0 PID: 3305 Comm: kworker/u64:13 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [517.634304] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [517.634311] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [517.634320] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [517.634349] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [517.634816] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [517.634826] RSP: 0018:ffffc90003053ca0 EFLAGS: 00010246 <4> [517.634838] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [517.634845] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [517.634852] RBP: ffffc90003053db0 R08: 0000000000000000 R09: 0000000000000000 <4> [517.634858] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [517.634865] R13: ffff888104cd0010 R14: ffff88812b1b0000 R15: 00000000ffffffc2 <4> [517.634872] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000 <4> [517.634880] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [517.634888] CR2: 000059da42793190 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [517.634896] PKRU: 55555554 <4> [517.634902] Call Trace: <4> [517.634909] <TASK> <4> [517.634930] ? lock_sync+0x100/0x100 <4> [517.634957] ? __pfx_autoremove_wake_function+0x10/0x10 <4> [517.634985] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [517.635014] process_one_work+0x239/0x760 <4> [517.635048] worker_thread+0x200/0x3f0 <4> [517.635062] ? __pfx_worker_thread+0x10/0x10 <4> [517.635074] kthread+0x10d/0x150 <4> [517.635090] ? __pfx_kthread+0x10/0x10 <4> [517.635109] ret_from_fork+0x3d4/0x480 <4> [517.635120] ? __pfx_kthread+0x10/0x10 <4> [517.635138] ret_from_fork_asm+0x1a/0x30 <4> [517.635178] </TASK> <4> [517.635184] irq event stamp: 473119 <4> [517.635191] hardirqs last enabled at (473125): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [517.635207] hardirqs last disabled at (473130): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [517.635219] softirqs last enabled at (472958): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [517.635232] softirqs last disabled at (472953): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [517.635244] ---[ end trace 0000000000000000 ]--- <7> [517.645398] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [517.645734] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [517.645873] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [517.645982] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [517.646091] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [517.646201] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [517.646311] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [517.646463] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [517.646577] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [517.646684] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [517.646789] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [517.646894] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [517.647024] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [517.647131] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [517.647233] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [517.647336] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [517.647471] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [517.647577] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [517.647684] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [517.647789] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [517.647893] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [517.647997] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [517.648102] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [517.648205] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [517.655249] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [517.655394] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [517.655531] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [517.655650] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [517.655755] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [517.655861] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [517.655965] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [517.656068] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [517.656172] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [517.656275] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [517.656399] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [517.656516] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [517.656645] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [517.656755] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [517.656859] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [517.656966] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [517.657099] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [517.657226] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [517.657343] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [517.657478] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [517.657582] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [517.657685] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [517.657789] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [517.658087] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.301039] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=3, flags=0x4 in no process [-1] <7> [518.301067] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.301555] ------------[ cut here ]------------ <4> [518.301562] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.301570] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [518.301982] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.302359] autofs4 [last unloaded: snd_hda_intel] <4> [518.302399] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.302415] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.302421] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.302430] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.302459] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.302849] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.302859] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [518.302873] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.302881] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.302888] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.302894] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.302900] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.302907] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [518.302917] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.302925] CR2: 0000799f62a10000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.302934] PKRU: 55555554 <4> [518.302941] Call Trace: <4> [518.302947] <TASK> <4> [518.302968] ? lock_sync+0x100/0x100 <4> [518.302993] ? lock_release+0xd0/0x2b0 <4> [518.303023] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.303052] process_one_work+0x239/0x760 <4> [518.303085] worker_thread+0x200/0x3f0 <4> [518.303099] ? __pfx_worker_thread+0x10/0x10 <4> [518.303112] kthread+0x10d/0x150 <4> [518.303128] ? __pfx_kthread+0x10/0x10 <4> [518.303147] ret_from_fork+0x3d4/0x480 <4> [518.303159] ? __pfx_kthread+0x10/0x10 <4> [518.303177] ret_from_fork_asm+0x1a/0x30 <4> [518.303216] </TASK> <4> [518.303222] irq event stamp: 661493 <4> [518.303229] hardirqs last enabled at (661499): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.303245] hardirqs last disabled at (661504): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.303257] softirqs last enabled at (660730): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.303271] softirqs last disabled at (660723): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.303283] ---[ end trace 0000000000000000 ]--- <6> [518.303293] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.303737] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.303789] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.303861] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.305062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.305158] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.305249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.305336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.305434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.305518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.305604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.305690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.305772] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.305865] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.305978] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.306989] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.317831] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.318269] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.320107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.320332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.320588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.320802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.321012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.321220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.321439] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.321645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.321840] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.322030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.322220] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.322423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.322607] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.322789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.322967] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.323144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.323319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.323513] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.323681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.323860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.324035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.324207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.324377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.324539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.324706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.324861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.325014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.325173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.325330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.325493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.325640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.325786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.325938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.326092] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.326242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.326385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.326522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.326653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.326781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.326907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.327039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.327167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.327290] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.327425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.327557] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.327678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.327793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.327906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.328017] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.328123] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.328231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.328344] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.328463] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.328569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.328669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.328776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.328881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.328981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.329083] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.329693] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.329699] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=3, flags=0x4 in no process [-1] <7> [518.329702] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.329772] ------------[ cut here ]------------ <4> [518.329774] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.329776] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [518.329861] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.329935] autofs4 [last unloaded: snd_hda_intel] <4> [518.329940] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.329943] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.329945] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.329946] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.329952] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.330032] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.330034] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [518.330036] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.330038] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.330039] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.330041] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.330042] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.330044] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [518.330046] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.330047] CR2: 0000799f62a10000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.330049] PKRU: 55555554 <4> [518.330050] Call Trace: <4> [518.330052] <TASK> <4> [518.330056] ? lock_sync+0x100/0x100 <4> [518.330061] ? lock_release+0xd0/0x2b0 <4> [518.330068] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.330074] process_one_work+0x239/0x760 <4> [518.330081] worker_thread+0x200/0x3f0 <4> [518.330084] ? __pfx_worker_thread+0x10/0x10 <4> [518.330087] kthread+0x10d/0x150 <4> [518.330090] ? __pfx_kthread+0x10/0x10 <4> [518.330094] ret_from_fork+0x3d4/0x480 <4> [518.330097] ? __pfx_kthread+0x10/0x10 <4> [518.330101] ret_from_fork_asm+0x1a/0x30 <4> [518.330109] </TASK> <4> [518.330110] irq event stamp: 665529 <4> [518.330112] hardirqs last enabled at (665535): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.330115] hardirqs last disabled at (665540): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.330118] softirqs last enabled at (664366): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.330120] softirqs last disabled at (664359): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.330123] ---[ end trace 0000000000000000 ]--- <6> [518.330125] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.330204] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.330211] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.331654] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.331778] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.332135] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.332402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.332500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.332599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.332694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.332790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.332884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.332977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.333070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.333158] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.333258] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.333384] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.334397] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.345124] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.345380] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.346489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.346567] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.346645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.346724] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.346802] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.346879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.346954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.347029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.347105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.347179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.347254] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.347329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.347414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.347489] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.347563] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.347638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.347712] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.347786] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.347861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.347943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.348027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.348109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.348190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.348268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.348345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.348435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.348512] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.348591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.348671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.348748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.348828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.348906] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.348990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.349077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.349164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.349247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.349328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.349415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.349494] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.349569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.349649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.349730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.349812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.349890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.349971] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.350048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.350125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.350202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.350279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.350354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.350441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.350521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.350602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.350683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.350762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.350842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.350918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.350994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.351075] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.351525] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.351530] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=3, flags=0x4 in no process [-1] <7> [518.351533] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.351593] ------------[ cut here ]------------ <4> [518.351594] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.351596] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [518.351668] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.351742] autofs4 [last unloaded: snd_hda_intel] <4> [518.351747] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.351750] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.351751] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.351753] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.351759] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.351830] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.351832] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [518.351834] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.351836] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.351837] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.351838] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.351839] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.351841] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [518.351843] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.351844] CR2: 0000799f62a10000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.351845] PKRU: 55555554 <4> [518.351847] Call Trace: <4> [518.351848] <TASK> <4> [518.351852] ? lock_sync+0x100/0x100 <4> [518.351857] ? lock_release+0xd0/0x2b0 <4> [518.351863] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.351868] process_one_work+0x239/0x760 <4> [518.351874] worker_thread+0x200/0x3f0 <4> [518.351877] ? __pfx_worker_thread+0x10/0x10 <4> [518.351880] kthread+0x10d/0x150 <4> [518.351882] ? __pfx_kthread+0x10/0x10 <4> [518.351886] ret_from_fork+0x3d4/0x480 <4> [518.351888] ? __pfx_kthread+0x10/0x10 <4> [518.351891] ret_from_fork_asm+0x1a/0x30 <4> [518.351899] </TASK> <4> [518.351900] irq event stamp: 669525 <4> [518.351901] hardirqs last enabled at (669531): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.351904] hardirqs last disabled at (669536): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.351906] softirqs last enabled at (668728): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.351909] softirqs last disabled at (668717): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.351911] ---[ end trace 0000000000000000 ]--- <7> [518.353205] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.353354] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.360627] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.361017] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.361145] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.361266] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.361395] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.361515] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.361632] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.361747] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.361861] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.361977] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.362090] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.362205] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.362348] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.362487] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.362601] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.362714] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.362831] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.362946] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.363059] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.363174] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.363286] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.363405] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.363517] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.363629] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.370798] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.370937] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.371055] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.371171] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.371286] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.371554] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.371673] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.371791] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.371907] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.372024] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.372140] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.372254] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.372625] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.372758] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.372876] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.372992] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.373107] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.373220] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.373334] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.373462] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.373576] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.373690] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.373804] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.373917] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.416195] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=1, flags=0x4 in no process [-1] <7> [518.416221] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.416668] ------------[ cut here ]------------ <4> [518.416681] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.416696] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#2: kworker/u64:8/990 <4> [518.417310] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.418032] autofs4 [last unloaded: snd_hda_intel] <4> [518.418069] CPU: 2 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.418095] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.418107] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.418121] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.418165] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.418839] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.418856] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [518.418879] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.418893] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.418907] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.418919] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.418931] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.418944] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [518.418960] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.418974] CR2: 000000c0004cb000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.418988] PKRU: 55555554 <4> [518.418999] Call Trace: <4> [518.419010] <TASK> <4> [518.419050] ? lock_sync+0x100/0x100 <4> [518.419095] ? lock_release+0xd0/0x2b0 <4> [518.419146] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.419198] process_one_work+0x239/0x760 <4> [518.419259] worker_thread+0x200/0x3f0 <4> [518.419287] ? __pfx_worker_thread+0x10/0x10 <4> [518.419310] kthread+0x10d/0x150 <4> [518.419334] ? __pfx_kthread+0x10/0x10 <4> [518.419420] ret_from_fork+0x3d4/0x480 <4> [518.419436] ? __pfx_kthread+0x10/0x10 <4> [518.419466] ret_from_fork_asm+0x1a/0x30 <4> [518.419538] </TASK> <4> [518.419549] irq event stamp: 336917 <4> [518.419561] hardirqs last enabled at (336923): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.419584] hardirqs last disabled at (336928): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.419605] softirqs last enabled at (336892): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.419625] softirqs last disabled at (336869): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.419645] ---[ end trace 0000000000000000 ]--- <6> [518.419662] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.420198] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.420256] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.420738] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.421733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.422203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.422761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.423216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.423656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.424054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.424454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.424817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.425155] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.425602] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.425994] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.427464] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.437447] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.437708] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.439063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.439149] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.439230] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.439313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.439409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.439500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.439581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.439665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.439746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.439827] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.439905] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.439985] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.440063] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.440141] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.440224] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.440304] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.440389] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.440473] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.440549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.440629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.440710] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.440791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.440875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.440956] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.441035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.441113] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.441191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.441271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.441351] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.441447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.441537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.441617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.441701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.441785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.441869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.441950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.442029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.442107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.442190] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.442269] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.442350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.442451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.442541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.442618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.442699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.442780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.442861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.442941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.443020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.443101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.443180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.443261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.443340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.443430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.443517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.443597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.443672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.443750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.443833] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.444124] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.444129] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=1, flags=0x4 in no process [-1] <7> [518.444132] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.444191] ------------[ cut here ]------------ <4> [518.444192] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.444194] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:8/990 <4> [518.444266] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.444334] autofs4 [last unloaded: snd_hda_intel] <4> [518.444339] CPU: 1 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.444342] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.444343] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.444345] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.444350] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.444438] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.444441] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [518.444445] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.444448] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.444450] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.444452] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.444455] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.444457] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.444460] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.444463] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.444465] PKRU: 55555554 <4> [518.444467] Call Trace: <4> [518.444469] <TASK> <4> [518.444476] ? lock_sync+0x100/0x100 <4> [518.444485] ? lock_release+0xd0/0x2b0 <4> [518.444493] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.444503] process_one_work+0x239/0x760 <4> [518.444515] worker_thread+0x200/0x3f0 <4> [518.444520] ? __pfx_worker_thread+0x10/0x10 <4> [518.444524] kthread+0x10d/0x150 <4> [518.444529] ? __pfx_kthread+0x10/0x10 <4> [518.444535] ret_from_fork+0x3d4/0x480 <4> [518.444539] ? __pfx_kthread+0x10/0x10 <4> [518.444545] ret_from_fork_asm+0x1a/0x30 <4> [518.444558] </TASK> <4> [518.444560] irq event stamp: 340953 <4> [518.444562] hardirqs last enabled at (340959): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.444567] hardirqs last disabled at (340964): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.444571] softirqs last enabled at (339798): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.444575] softirqs last disabled at (339791): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.444579] ---[ end trace 0000000000000000 ]--- <6> [518.444582] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.444661] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <5> [518.444669] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=5, flags=0x0 in xe_evict [6588] <7> [518.444671] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <6> [518.444744] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.445155] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.445409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.445501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.445594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.445683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.445770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.445857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.445943] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.446032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.446115] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.446212] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.446333] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.447356] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.457370] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.457621] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.459156] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.459229] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.459301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.459381] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.459466] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.459545] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.459625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.459704] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.459782] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.459857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.459932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.460006] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.460080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.460153] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.460226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.460300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.460383] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.460468] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.460542] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.460624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.460705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.460786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.460869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.460947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.461024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.461101] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.461177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.461256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.461336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.461427] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.461516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.461596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.461684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.461771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.461856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.461938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.462019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.462097] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.462175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.462251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.462335] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.462431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.462522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.462602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.462685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.462763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.462841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.462917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.462996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.463072] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.463148] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.463228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.463306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.463393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.463484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.463566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.463645] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.463723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.463804] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.464066] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.464070] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967170, lrc_seqno=4294967170, guc_id=1, flags=0x4 in no process [-1] <7> [518.464072] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.464132] ------------[ cut here ]------------ <4> [518.464133] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.464134] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:8/990 <4> [518.464205] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.464271] autofs4 [last unloaded: snd_hda_intel] <4> [518.464275] CPU: 1 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.464278] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.464279] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.464281] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.464286] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.464356] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.464358] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [518.464370] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.464373] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.464375] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.464378] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.464380] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.464382] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.464385] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.464388] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.464391] PKRU: 55555554 <4> [518.464393] Call Trace: <4> [518.464395] <TASK> <4> [518.464402] ? lock_sync+0x100/0x100 <4> [518.464410] ? lock_release+0xd0/0x2b0 <4> [518.464419] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.464430] process_one_work+0x239/0x760 <4> [518.464441] worker_thread+0x200/0x3f0 <4> [518.464446] ? __pfx_worker_thread+0x10/0x10 <4> [518.464450] kthread+0x10d/0x150 <4> [518.464455] ? __pfx_kthread+0x10/0x10 <4> [518.464461] ret_from_fork+0x3d4/0x480 <4> [518.464464] ? __pfx_kthread+0x10/0x10 <4> [518.464470] ret_from_fork_asm+0x1a/0x30 <4> [518.464483] </TASK> <4> [518.464485] irq event stamp: 344985 <4> [518.464488] hardirqs last enabled at (344991): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.464492] hardirqs last disabled at (344996): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.464496] softirqs last enabled at (344838): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.464500] softirqs last disabled at (344827): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.464503] ---[ end trace 0000000000000000 ]--- <7> [518.465892] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.466007] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.472644] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.472860] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.472984] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.473108] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.473226] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.473345] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.473478] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.473593] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.473709] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.473851] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.474213] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.474338] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.474493] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.474618] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.474734] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.474847] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.474960] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.475075] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.475192] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.475308] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.475430] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.475545] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.475658] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.475770] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.482864] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.483006] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.483122] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.483235] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.483349] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.483480] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.483595] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.483711] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.483830] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.483949] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.484067] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.484182] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.484318] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.484446] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.484558] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.484670] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.484783] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.484896] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.485014] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.485131] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.485246] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.485364] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.485479] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.485594] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.528520] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=3, flags=0x4 in no process [-1] <7> [518.528546] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.528872] ------------[ cut here ]------------ <4> [518.528880] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.528888] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [518.529273] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.529668] autofs4 [last unloaded: snd_hda_intel] <4> [518.529693] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.529709] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.529716] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.529725] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.529752] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.530125] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.530136] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [518.530149] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.530157] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.530165] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.530172] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.530179] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.530187] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [518.530195] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.530203] CR2: 0000799f62a10000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.530212] PKRU: 55555554 <4> [518.530218] Call Trace: <4> [518.530224] <TASK> <4> [518.530245] ? lock_sync+0x100/0x100 <4> [518.530269] ? lock_release+0xd0/0x2b0 <4> [518.530297] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.530327] process_one_work+0x239/0x760 <4> [518.530387] worker_thread+0x200/0x3f0 <4> [518.530403] ? __pfx_worker_thread+0x10/0x10 <4> [518.530416] kthread+0x10d/0x150 <4> [518.530431] ? __pfx_kthread+0x10/0x10 <4> [518.530451] ret_from_fork+0x3d4/0x480 <4> [518.530462] ? __pfx_kthread+0x10/0x10 <4> [518.530481] ret_from_fork_asm+0x1a/0x30 <4> [518.530520] </TASK> <4> [518.530526] irq event stamp: 672555 <4> [518.530533] hardirqs last enabled at (672561): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.530547] hardirqs last disabled at (672566): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.530559] softirqs last enabled at (671942): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.530572] softirqs last disabled at (671931): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.530583] ---[ end trace 0000000000000000 ]--- <6> [518.530593] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.530958] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.530989] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.531440] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.532205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.532734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.533202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.533672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.534070] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.534453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.534824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.535171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.535557] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.535938] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.536320] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.537809] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.548754] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.549043] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.550099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.550178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.550255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.550332] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.550420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.550496] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.550573] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.550649] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.550723] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.550798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.550873] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.550949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.551024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.551099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.551176] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.551253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.551330] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.551415] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.551491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.551574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.551656] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.551739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.551821] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.551901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.551979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.552056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.552133] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.552214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.552295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.552377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.552457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.552536] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.552621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.552705] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.552793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.552878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.552962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.553041] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.553120] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.553197] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.553279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.553358] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.553450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.553533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.553618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.553698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.553776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.553853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.553931] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.554007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.554082] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.554162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.554240] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.554320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.554404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.554484] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.554561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.554638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.554716] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.554974] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.554978] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=3, flags=0x4 in no process [-1] <7> [518.554980] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.555039] ------------[ cut here ]------------ <4> [518.555040] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.555041] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [518.555113] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.555178] autofs4 [last unloaded: snd_hda_intel] <4> [518.555181] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.555184] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.555185] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.555187] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.555192] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.555261] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.555263] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [518.555265] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.555267] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.555268] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.555269] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.555270] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.555272] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [518.555273] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.555275] CR2: 0000799f62a10000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [518.555276] PKRU: 55555554 <4> [518.555278] Call Trace: <4> [518.555279] <TASK> <4> [518.555283] ? lock_sync+0x100/0x100 <4> [518.555287] ? lock_release+0xd0/0x2b0 <4> [518.555293] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.555298] process_one_work+0x239/0x760 <4> [518.555305] worker_thread+0x200/0x3f0 <4> [518.555307] ? __pfx_worker_thread+0x10/0x10 <4> [518.555310] kthread+0x10d/0x150 <4> [518.555313] ? __pfx_kthread+0x10/0x10 <4> [518.555316] ret_from_fork+0x3d4/0x480 <4> [518.555319] ? __pfx_kthread+0x10/0x10 <4> [518.555322] ret_from_fork_asm+0x1a/0x30 <4> [518.555330] </TASK> <4> [518.555331] irq event stamp: 676611 <4> [518.555332] hardirqs last enabled at (676617): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.555335] hardirqs last disabled at (676622): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.555337] softirqs last enabled at (675274): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.555339] softirqs last disabled at (675267): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.555342] ---[ end trace 0000000000000000 ]--- <6> [518.555343] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.555427] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <5> [518.555435] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=8, flags=0x0 in xe_evict [6588] <7> [518.555438] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <6> [518.555506] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.555913] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.556156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.556246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.556338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.556436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.556522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.556608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.556694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.556780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.556863] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.556959] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.557075] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.558081] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.568716] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.568959] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.570023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.570102] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.570178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.570257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.570334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.570423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.570501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.570577] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.570654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.570729] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.570804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.570879] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.570954] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.571029] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.571104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.571178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.571253] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.571326] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.571419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.571503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.571587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.571668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.571748] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.571825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.571902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.571978] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.572055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.572135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.572214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.572293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.572374] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.572452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.572540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.572629] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.572715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.572799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.572880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.572959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.573036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.573112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.573192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.573270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.573352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.573445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.573527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.573607] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.573685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.573761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.573839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.573916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.573995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.574076] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.574156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.574235] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.574313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.574400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.574476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.574552] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.574630] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.574858] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.574877] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=3, flags=0x4 in no process [-1] <7> [518.574885] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.575035] ------------[ cut here ]------------ <4> [518.575037] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.575040] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#12: kworker/u64:14/5663 <4> [518.575159] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.575261] autofs4 [last unloaded: snd_hda_intel] <4> [518.575267] CPU: 12 UID: 0 PID: 5663 Comm: kworker/u64:14 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.575272] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.575273] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.575276] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.575285] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.575420] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.575423] RSP: 0018:ffffc90003473ca0 EFLAGS: 00010246 <4> [518.575427] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.575429] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.575431] RBP: ffffc90003473db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.575433] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.575435] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.575437] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [518.575440] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.575442] CR2: 000071410741f260 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [518.575444] PKRU: 55555554 <4> [518.575446] Call Trace: <4> [518.575448] <TASK> <4> [518.575455] ? lock_sync+0x100/0x100 <4> [518.575464] ? lock_release+0xd0/0x2b0 <4> [518.575474] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.575483] process_one_work+0x239/0x760 <4> [518.575494] worker_thread+0x200/0x3f0 <4> [518.575498] ? __pfx_worker_thread+0x10/0x10 <4> [518.575501] kthread+0x10d/0x150 <4> [518.575506] ? __pfx_kthread+0x10/0x10 <4> [518.575511] ret_from_fork+0x3d4/0x480 <4> [518.575514] ? __pfx_kthread+0x10/0x10 <4> [518.575519] ret_from_fork_asm+0x1a/0x30 <4> [518.575532] </TASK> <4> [518.575533] irq event stamp: 158741 <4> [518.575535] hardirqs last enabled at (158747): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.575539] hardirqs last disabled at (158752): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.575542] softirqs last enabled at (158578): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.575546] softirqs last disabled at (158563): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.575549] ---[ end trace 0000000000000000 ]--- <7> [518.575850] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.575960] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.583257] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.583533] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.583657] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.583776] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.583894] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.584013] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.584132] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.584249] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.584373] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.584492] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.584611] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.584726] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.584865] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.584981] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.585094] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.585206] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.585318] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.585456] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.585574] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.585690] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.585805] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.586061] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.586176] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.586288] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.593589] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.593729] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.593846] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.593960] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.594074] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.594188] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.594305] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.594554] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.594675] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.594791] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.594907] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.595022] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.595158] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.595276] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.595478] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.595594] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.595710] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.595825] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.595941] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.596053] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.596167] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.596279] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.596419] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.596535] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.639586] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=1, flags=0x4 in no process [-1] <7> [518.639618] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.639945] ------------[ cut here ]------------ <4> [518.639952] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.639960] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:14/5663 <4> [518.640351] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.640887] autofs4 [last unloaded: snd_hda_intel] <4> [518.640911] CPU: 1 UID: 0 PID: 5663 Comm: kworker/u64:14 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.640928] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.640935] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.640943] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.640973] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.641353] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.641393] RSP: 0018:ffffc90003473ca0 EFLAGS: 00010246 <4> [518.641414] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.641427] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.641440] RBP: ffffc90003473db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.641452] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.641463] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.641475] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.641490] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.641502] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.641516] PKRU: 55555554 <4> [518.641526] Call Trace: <4> [518.641536] <TASK> <4> [518.641560] ? lock_sync+0x100/0x100 <4> [518.641585] ? lock_release+0xd0/0x2b0 <4> [518.641614] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.641644] process_one_work+0x239/0x760 <4> [518.641676] worker_thread+0x200/0x3f0 <4> [518.641691] ? __pfx_worker_thread+0x10/0x10 <4> [518.641704] kthread+0x10d/0x150 <4> [518.641718] ? __pfx_kthread+0x10/0x10 <4> [518.641738] ret_from_fork+0x3d4/0x480 <4> [518.641750] ? __pfx_kthread+0x10/0x10 <4> [518.641768] ret_from_fork_asm+0x1a/0x30 <4> [518.641805] </TASK> <4> [518.641811] irq event stamp: 161481 <4> [518.641818] hardirqs last enabled at (161487): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.641834] hardirqs last disabled at (161492): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.641845] softirqs last enabled at (159002): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.641858] softirqs last disabled at (158997): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.641870] ---[ end trace 0000000000000000 ]--- <6> [518.641881] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.642268] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.642305] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.642787] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.643597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.644066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.644622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.644712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.644812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.644917] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.645005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.645098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.645185] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.645281] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.645407] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.646424] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.656382] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.656663] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.657741] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.657858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.657972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.658084] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.658196] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.658300] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.658417] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.658519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.658645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.658755] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.658850] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.658949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.659047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.659144] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.659238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.659327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.659437] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.659526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.659614] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.659712] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.659806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.659897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.659990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.660081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.660171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.660253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.660339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.660444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.660528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.660609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.660690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.660770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.660853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.660938] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.661021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.661102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.661181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.661289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.661392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.661476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.661560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.661643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.661725] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.661804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.661887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.661966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.662044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.662121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.662200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.662277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.662357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.662457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.662538] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.662619] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.662697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.662776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.662851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.662932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.663017] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.663300] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.663304] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=1, flags=0x4 in no process [-1] <7> [518.663307] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.664441] ------------[ cut here ]------------ <4> [518.664445] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.664450] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#0: kworker/u64:14/5663 <4> [518.664609] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.664701] autofs4 [last unloaded: snd_hda_intel] <4> [518.664708] CPU: 0 UID: 0 PID: 5663 Comm: kworker/u64:14 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.664713] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.664715] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.664717] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.664725] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <7> [518.664717] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <4> [518.664844] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.664847] RSP: 0018:ffffc90003473ca0 EFLAGS: 00010246 <4> [518.664850] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.664852] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.664854] RBP: ffffc90003473db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.664856] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.664858] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.664860] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [518.664862] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.664864] CR2: 0000799f62a12000 CR3: 0000000123a85004 CR4: 0000000000f72ef0 <4> [518.664867] PKRU: 55555554 <4> [518.664868] Call Trace: <4> [518.664870] <TASK> <4> [518.664882] ? lock_sync+0x100/0x100 <4> [518.664890] ? lock_release+0xd0/0x2b0 <4> [518.664898] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.664907] process_one_work+0x239/0x760 <4> [518.664917] worker_thread+0x200/0x3f0 <4> [518.664922] ? __pfx_worker_thread+0x10/0x10 <4> [518.664926] kthread+0x10d/0x150 <4> [518.664931] ? __pfx_kthread+0x10/0x10 <4> [518.664937] ret_from_fork+0x3d4/0x480 <4> [518.664940] ? __pfx_kthread+0x10/0x10 <4> [518.664946] ret_from_fork_asm+0x1a/0x30 <7> [518.664880] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <4> [518.664959] </TASK> <4> [518.664960] irq event stamp: 165489 <4> [518.664962] hardirqs last enabled at (165495): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.664965] hardirqs last disabled at (165500): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.664968] softirqs last enabled at (164128): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.664971] softirqs last disabled at (164121): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.664974] ---[ end trace 0000000000000000 ]--- <6> [518.664976] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.665071] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.665078] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.665614] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.665861] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.665954] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.666048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.666136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.666224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.666310] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.666408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.666498] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.666581] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.666678] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.666799] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.667814] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.678378] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.678629] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.679657] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.679736] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.679813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.679891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.679970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.680047] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.680127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.680205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.680283] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.680358] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.680450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.680525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.680601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.680676] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.680750] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.680823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.680897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.680971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.681044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.681125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.681207] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.681288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.681379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.681459] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.681539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.681620] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.681698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.681780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.681860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.681939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.682019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.682103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.682191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.682276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.682367] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.682452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.682533] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.682612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.682688] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.682765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.682844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.682923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.682999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.683075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.683160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.683239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.683319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.683407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.683489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.683565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.683641] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.683721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.683799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.683875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.683952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.684029] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.684105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.684181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.684261] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.684527] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.684532] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967171, lrc_seqno=4294967171, guc_id=1, flags=0x4 in no process [-1] <7> [518.684534] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.684593] ------------[ cut here ]------------ <4> [518.684594] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.684595] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#0: kworker/u64:14/5663 <4> [518.684669] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.684734] autofs4 [last unloaded: snd_hda_intel] <4> [518.684738] CPU: 0 UID: 0 PID: 5663 Comm: kworker/u64:14 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.684741] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.684743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.684744] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.684749] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.684818] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.684820] RSP: 0018:ffffc90003473ca0 EFLAGS: 00010246 <4> [518.684822] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.684824] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.684825] RBP: ffffc90003473db0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.684826] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.684827] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.684829] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [518.684830] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.684832] CR2: 0000799f62a12000 CR3: 0000000123a85004 CR4: 0000000000f72ef0 <4> [518.684833] PKRU: 55555554 <4> [518.684835] Call Trace: <4> [518.684836] <TASK> <4> [518.684840] ? lock_sync+0x100/0x100 <4> [518.684844] ? lock_release+0xd0/0x2b0 <4> [518.684849] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.684855] process_one_work+0x239/0x760 <4> [518.684861] worker_thread+0x200/0x3f0 <4> [518.684864] ? __pfx_worker_thread+0x10/0x10 <4> [518.684866] kthread+0x10d/0x150 <4> [518.684869] ? __pfx_kthread+0x10/0x10 <4> [518.684873] ret_from_fork+0x3d4/0x480 <4> [518.684875] ? __pfx_kthread+0x10/0x10 <4> [518.684878] ret_from_fork_asm+0x1a/0x30 <4> [518.684886] </TASK> <4> [518.684887] irq event stamp: 169447 <4> [518.684888] hardirqs last enabled at (169453): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.684891] hardirqs last disabled at (169458): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.684894] softirqs last enabled at (168064): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.684897] softirqs last disabled at (168059): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.684899] ---[ end trace 0000000000000000 ]--- <7> [518.685965] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.686093] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.692497] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.692795] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.692921] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.693042] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.693160] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.693277] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.693423] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.693550] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.693666] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.693781] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.693897] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.694012] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.694152] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.694270] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.694408] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.694539] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.694652] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.694768] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.694884] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.694999] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.695113] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.695227] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.695340] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.695490] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.702629] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.702775] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.702892] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.703006] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.703120] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.703234] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.703348] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.703513] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.703754] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.703876] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.703996] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.704114] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.704253] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.704381] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.704509] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.704623] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.704736] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.704849] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.704962] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.705077] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.705193] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.705308] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.705461] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.705582] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.748433] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=3, flags=0x4 in no process [-1] <7> [518.748462] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.748790] ------------[ cut here ]------------ <4> [518.748797] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.748805] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.749189] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.749698] autofs4 [last unloaded: snd_hda_intel] <4> [518.749722] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.749739] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.749746] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.749754] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.749781] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.750149] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.750159] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.750173] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.750182] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.750190] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.750197] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.750204] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.750213] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.750222] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.750230] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.750240] PKRU: 55555554 <4> [518.750247] Call Trace: <4> [518.750254] <TASK> <4> [518.750275] ? lock_sync+0x100/0x100 <4> [518.750300] ? lock_release+0xd0/0x2b0 <4> [518.750330] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.750389] process_one_work+0x239/0x760 <4> [518.750448] worker_thread+0x200/0x3f0 <4> [518.750475] ? __pfx_worker_thread+0x10/0x10 <4> [518.750498] kthread+0x10d/0x150 <4> [518.750523] ? __pfx_kthread+0x10/0x10 <4> [518.750553] ret_from_fork+0x3d4/0x480 <4> [518.750571] ? __pfx_kthread+0x10/0x10 <4> [518.750602] ret_from_fork_asm+0x1a/0x30 <4> [518.750673] </TASK> <4> [518.750685] irq event stamp: 190087 <4> [518.750696] hardirqs last enabled at (190093): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.750718] hardirqs last disabled at (190098): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.750738] softirqs last enabled at (189602): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.750758] softirqs last disabled at (189597): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.750777] ---[ end trace 0000000000000000 ]--- <6> [518.750792] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.751198] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.751232] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.751820] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.752683] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.753142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.753745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.754189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.754686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.755127] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.755615] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.756067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.756537] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.757042] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.757612] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.759313] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.770416] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.770677] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.771781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.771854] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.771925] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.771998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.772071] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.772140] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.772211] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.772281] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.772353] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.772450] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.772538] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.772617] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.772693] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.772768] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.772842] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.772916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.772990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.773064] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.773137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.773219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.773301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.773393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.773490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.773570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.773649] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.773727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.773804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.773884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.773963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.774042] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.774121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.774198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.774283] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.774373] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.774465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.774546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.774628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.774711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.774794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.774872] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.774952] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.775033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.775115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.775194] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.775277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.775355] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.775452] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.775539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.775618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.775694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.775773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.775857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.775937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.776016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.776093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.776173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.776249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.776325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.776421] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.776688] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.776693] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=3, flags=0x4 in no process [-1] <7> [518.776695] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.776755] ------------[ cut here ]------------ <4> [518.776757] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.776758] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.776830] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.776899] autofs4 [last unloaded: snd_hda_intel] <4> [518.776903] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.776906] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.776908] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.776909] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.776915] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.776985] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.776987] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.776990] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.776991] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.776993] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.776994] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.776996] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.776997] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.776999] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.777001] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.777002] PKRU: 55555554 <4> [518.777004] Call Trace: <4> [518.777005] <TASK> <4> [518.777009] ? lock_sync+0x100/0x100 <4> [518.777014] ? lock_release+0xd0/0x2b0 <4> [518.777020] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.777025] process_one_work+0x239/0x760 <4> [518.777032] worker_thread+0x200/0x3f0 <4> [518.777035] ? __pfx_worker_thread+0x10/0x10 <4> [518.777038] kthread+0x10d/0x150 <4> [518.777041] ? __pfx_kthread+0x10/0x10 <4> [518.777045] ret_from_fork+0x3d4/0x480 <4> [518.777047] ? __pfx_kthread+0x10/0x10 <4> [518.777051] ret_from_fork_asm+0x1a/0x30 <4> [518.777059] </TASK> <4> [518.777060] irq event stamp: 194165 <4> [518.777062] hardirqs last enabled at (194171): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.777065] hardirqs last disabled at (194176): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.777067] softirqs last enabled at (193042): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.777070] softirqs last disabled at (193031): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.777073] ---[ end trace 0000000000000000 ]--- <6> [518.777075] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.777144] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.777151] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.777562] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.777837] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.777926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.778019] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.778108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.778195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.778282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.778385] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.778493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.778576] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.778671] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.778788] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.779829] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.790368] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.790623] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.791733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.791813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.791891] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.791969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.792045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.792121] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.792197] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.792273] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.792348] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.792442] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.792526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.792601] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.792674] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.792751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.792830] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.792907] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.792983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.793059] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.793135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.793217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.793299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.793386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.793479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.793561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.793639] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.793717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.793795] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.793874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.793955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.794033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.794112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.794191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.794275] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.794364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.794461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.794541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.794622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.794699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.794777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.794856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.794941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.795024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.795103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.795182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.795264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.795342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.795434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.795524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.795604] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.795686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.795767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.795849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.795928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.796006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.796084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.796162] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.796242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.796323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.796422] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.796692] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.796697] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=3, flags=0x4 in no process [-1] <7> [518.796699] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.796758] ------------[ cut here ]------------ <4> [518.796759] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.796760] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.796831] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.796909] autofs4 [last unloaded: snd_hda_intel] <4> [518.796913] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.796916] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.796917] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.796919] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.796924] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.796994] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.796995] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.796998] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.796999] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.797001] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.797002] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.797003] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.797004] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.797006] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.797007] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.797009] PKRU: 55555554 <4> [518.797010] Call Trace: <4> [518.797011] <TASK> <4> [518.797015] ? lock_sync+0x100/0x100 <4> [518.797019] ? lock_release+0xd0/0x2b0 <4> [518.797025] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.797031] process_one_work+0x239/0x760 <4> [518.797037] worker_thread+0x200/0x3f0 <4> [518.797040] ? __pfx_worker_thread+0x10/0x10 <4> [518.797042] kthread+0x10d/0x150 <4> [518.797045] ? __pfx_kthread+0x10/0x10 <4> [518.797049] ret_from_fork+0x3d4/0x480 <4> [518.797051] ? __pfx_kthread+0x10/0x10 <4> [518.797054] ret_from_fork_asm+0x1a/0x30 <4> [518.797061] </TASK> <4> [518.797062] irq event stamp: 198177 <4> [518.797064] hardirqs last enabled at (198183): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.797066] hardirqs last disabled at (198188): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.797069] softirqs last enabled at (197054): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.797071] softirqs last disabled at (197043): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.797073] ---[ end trace 0000000000000000 ]--- <7> [518.798693] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.798811] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.805239] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.805634] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.805765] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.805886] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.806005] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.806123] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.806239] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.806355] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.806516] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.806634] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.806755] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.806874] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.807017] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.807134] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.807248] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.807367] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.807504] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.807618] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.807730] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.807843] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.807955] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.808067] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.808179] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.808293] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.815457] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.815603] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.815721] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.815836] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.815949] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.816064] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.816177] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.816290] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.816448] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.816586] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.816705] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.816820] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.816957] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.817073] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.817187] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.817298] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.817446] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.817562] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.817675] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.817855] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.817971] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.818087] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.818202] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.818315] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.860174] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=1, flags=0x4 in no process [-1] <7> [518.860190] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.860592] ------------[ cut here ]------------ <4> [518.860603] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.860616] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.861093] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.861554] autofs4 [last unloaded: snd_hda_intel] <4> [518.861591] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.861607] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.861614] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.861624] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.861650] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.862015] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.862024] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.862036] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.862043] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.862050] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.862057] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.862063] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.862070] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.862079] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.862086] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.862094] PKRU: 55555554 <4> [518.862100] Call Trace: <4> [518.862106] <TASK> <4> [518.862126] ? lock_sync+0x100/0x100 <4> [518.862149] ? lock_release+0xd0/0x2b0 <4> [518.862177] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.862206] process_one_work+0x239/0x760 <4> [518.862238] worker_thread+0x200/0x3f0 <4> [518.862252] ? __pfx_worker_thread+0x10/0x10 <4> [518.862265] kthread+0x10d/0x150 <4> [518.862280] ? __pfx_kthread+0x10/0x10 <4> [518.862299] ret_from_fork+0x3d4/0x480 <4> [518.862309] ? __pfx_kthread+0x10/0x10 <4> [518.862326] ret_from_fork_asm+0x1a/0x30 <4> [518.862397] </TASK> <4> [518.862404] irq event stamp: 201175 <4> [518.862410] hardirqs last enabled at (201181): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.862425] hardirqs last disabled at (201186): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.862438] softirqs last enabled at (200804): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.862451] softirqs last disabled at (200799): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.862463] ---[ end trace 0000000000000000 ]--- <6> [518.862474] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.862842] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.862874] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.863307] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.864151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.864672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.865147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.865672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.866094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.866551] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.866968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.867437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.867852] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.868324] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.868908] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.870555] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.881575] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.881833] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.883027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.883105] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.883182] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.883260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.883338] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.883428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.883509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.883586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.883663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.883742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.883823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.883903] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.883983] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.884060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.884137] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.884213] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.884287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.884365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.884443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.884528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.884610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.884689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.884770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.884848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.884925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.885004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.885083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.885164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.885244] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.885324] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.885423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.885508] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.885596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.885682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.885769] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.885850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.885930] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.886009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.886091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.886170] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.886252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.886331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.886422] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.886500] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.886586] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.886666] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.886746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.886823] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.886902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.886981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.887062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.887144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.887224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.887303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.887383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.887462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.887537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.887612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.887691] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.887971] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.887975] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=1, flags=0x4 in no process [-1] <7> [518.887978] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.888036] ------------[ cut here ]------------ <4> [518.888037] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.888039] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.888110] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.888177] autofs4 [last unloaded: snd_hda_intel] <4> [518.888181] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.888184] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.888185] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.888186] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.888191] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.888261] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.888262] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.888265] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.888266] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.888267] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.888268] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.888270] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.888271] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.888273] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.888274] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.888276] PKRU: 55555554 <4> [518.888277] Call Trace: <4> [518.888278] <TASK> <4> [518.888282] ? lock_sync+0x100/0x100 <4> [518.888286] ? lock_release+0xd0/0x2b0 <4> [518.888292] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.888297] process_one_work+0x239/0x760 <4> [518.888303] worker_thread+0x200/0x3f0 <4> [518.888306] ? __pfx_worker_thread+0x10/0x10 <4> [518.888308] kthread+0x10d/0x150 <4> [518.888312] ? __pfx_kthread+0x10/0x10 <4> [518.888315] ret_from_fork+0x3d4/0x480 <4> [518.888317] ? __pfx_kthread+0x10/0x10 <4> [518.888321] ret_from_fork_asm+0x1a/0x30 <4> [518.888328] </TASK> <4> [518.888329] irq event stamp: 205251 <4> [518.888330] hardirqs last enabled at (205257): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.888333] hardirqs last disabled at (205262): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.888335] softirqs last enabled at (204022): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.888338] softirqs last disabled at (204015): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.888340] ---[ end trace 0000000000000000 ]--- <6> [518.888342] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.889646] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.889653] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.889788] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.889896] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.890004] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.890248] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.890337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.890445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.890537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.890625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.890711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.890811] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.890899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.890982] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.891077] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.891195] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.892202] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.902366] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.902615] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.903744] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.903823] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.903900] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.903977] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.904057] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.904136] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.904214] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.904291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.904371] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.904446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.904522] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.904597] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.904672] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.904746] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.904825] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.904902] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.904979] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.905056] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.905131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.905215] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.905298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.905383] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.905465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.905547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.905628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.905706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.905784] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.905865] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.905945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.906024] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.906103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.906183] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.906268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.906352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.906451] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.906531] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.906611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.906695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.906777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.906857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.906939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.907018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.907096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.907174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.907255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.907333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.907429] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.907511] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.907591] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.907669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.907746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.907829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.907911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.907992] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.908071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.908150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.908228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.908306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.908392] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.908642] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.908646] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967172, lrc_seqno=4294967172, guc_id=1, flags=0x4 in no process [-1] <7> [518.908648] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.908708] ------------[ cut here ]------------ <4> [518.908709] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.908711] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.908784] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.908850] autofs4 [last unloaded: snd_hda_intel] <4> [518.908853] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.908856] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.908858] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.908859] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.908864] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.908935] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.908937] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.908939] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.908940] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.908942] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.908943] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.908944] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [518.908945] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.908947] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.908948] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.908950] PKRU: 55555554 <4> [518.908951] Call Trace: <4> [518.908952] <TASK> <4> [518.908956] ? lock_sync+0x100/0x100 <4> [518.908960] ? lock_release+0xd0/0x2b0 <4> [518.908966] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.908971] process_one_work+0x239/0x760 <4> [518.908978] worker_thread+0x200/0x3f0 <4> [518.908981] ? __pfx_worker_thread+0x10/0x10 <4> [518.908983] kthread+0x10d/0x150 <4> [518.908986] ? __pfx_kthread+0x10/0x10 <4> [518.908990] ret_from_fork+0x3d4/0x480 <4> [518.908992] ? __pfx_kthread+0x10/0x10 <4> [518.908995] ret_from_fork_asm+0x1a/0x30 <4> [518.909003] </TASK> <4> [518.909004] irq event stamp: 209283 <4> [518.909005] hardirqs last enabled at (209289): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.909008] hardirqs last disabled at (209294): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.909010] softirqs last enabled at (208082): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.909012] softirqs last disabled at (208075): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.909015] ---[ end trace 0000000000000000 ]--- <7> [518.910060] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [518.910180] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [518.916873] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.917434] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.917569] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.917694] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.917815] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.917932] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.918050] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.918166] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.918281] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.918404] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.918521] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.918634] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.918777] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.918896] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.919011] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.919127] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.919242] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.919357] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.919489] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.919603] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.919716] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.919830] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.919941] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.920053] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.927128] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.927275] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.927398] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.927514] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.927629] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.927747] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.927865] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.927979] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.928094] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.928208] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.928322] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.928446] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [518.928582] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [518.928699] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [518.928812] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [518.928925] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [518.929036] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [518.929150] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [518.929265] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [518.929387] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [518.929501] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [518.929612] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [518.929724] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [518.929838] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [518.972487] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=3, flags=0x4 in no process [-1] <7> [518.972514] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.972845] ------------[ cut here ]------------ <4> [518.972852] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.972860] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:17/5666 <4> [518.973250] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.973673] autofs4 [last unloaded: snd_hda_intel] <4> [518.973697] CPU: 1 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.973714] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.973721] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.973730] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.973758] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [518.974125] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [518.974135] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [518.974150] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [518.974158] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [518.974165] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [518.974172] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [518.974179] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [518.974188] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [518.974197] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [518.974204] CR2: 0000000625eb2008 CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [518.974213] PKRU: 55555554 <4> [518.974220] Call Trace: <4> [518.974226] <TASK> <4> [518.974248] ? lock_sync+0x100/0x100 <4> [518.974274] ? lock_release+0xd0/0x2b0 <4> [518.974303] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [518.974333] process_one_work+0x239/0x760 <4> [518.974411] worker_thread+0x200/0x3f0 <4> [518.974438] ? __pfx_worker_thread+0x10/0x10 <4> [518.974457] kthread+0x10d/0x150 <4> [518.974473] ? __pfx_kthread+0x10/0x10 <4> [518.974492] ret_from_fork+0x3d4/0x480 <4> [518.974503] ? __pfx_kthread+0x10/0x10 <4> [518.974520] ret_from_fork_asm+0x1a/0x30 <4> [518.974560] </TASK> <4> [518.974566] irq event stamp: 211225 <4> [518.974573] hardirqs last enabled at (211231): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [518.974588] hardirqs last disabled at (211236): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [518.974599] softirqs last enabled at (211176): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.974613] softirqs last disabled at (211171): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [518.974626] ---[ end trace 0000000000000000 ]--- <6> [518.974637] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [518.975006] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [518.975040] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [518.975490] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [518.976258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [518.976806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [518.977272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [518.977810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [518.978210] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [518.978625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [518.979011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [518.979377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [518.979710] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [518.980090] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [518.980468] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [518.982483] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [518.993369] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [518.993641] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [518.994757] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [518.994846] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [518.994930] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [518.995014] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [518.995100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [518.995185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [518.995267] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [518.995347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [518.995447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [518.995528] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [518.995609] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [518.995690] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [518.995771] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [518.995855] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [518.995939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [518.996022] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [518.996104] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [518.996184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [518.996264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [518.996354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [518.996453] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [518.996535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [518.996618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [518.996699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [518.996780] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [518.996859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [518.996937] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [518.997018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [518.997099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [518.997178] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [518.997257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [518.997336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [518.997430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [518.997513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [518.997597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [518.997677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [518.997756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [518.997832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [518.997908] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [518.997983] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [518.998063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [518.998145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [518.998226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [518.998305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [518.998391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [518.998469] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [518.998546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [518.998622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [518.998699] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [518.998775] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [518.998852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [518.998932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [518.999010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [518.999093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [518.999173] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [518.999252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [518.999329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [518.999413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [518.999493] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [518.999766] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [518.999771] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=3, flags=0x4 in no process [-1] <7> [518.999774] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [518.999833] ------------[ cut here ]------------ <4> [518.999834] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [518.999836] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#11: kworker/u64:17/5666 <4> [518.999907] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [518.999975] autofs4 [last unloaded: snd_hda_intel] <4> [518.999980] CPU: 11 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [518.999983] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [518.999984] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [518.999985] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [518.999991] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.000062] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.000064] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [519.000066] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.000068] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.000069] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.000070] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.000071] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.000073] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000 <4> [519.000074] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.000076] CR2: 000071e234f6d000 CR3: 0000000123a85005 CR4: 0000000000f72ef0 <4> [519.000077] PKRU: 55555554 <4> [519.000078] Call Trace: <4> [519.000080] <TASK> <4> [519.000084] ? lock_sync+0x100/0x100 <4> [519.000089] ? lock_release+0xd0/0x2b0 <4> [519.000094] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.000100] process_one_work+0x239/0x760 <4> [519.000107] worker_thread+0x200/0x3f0 <4> [519.000110] ? __pfx_worker_thread+0x10/0x10 <4> [519.000112] kthread+0x10d/0x150 <4> [519.000116] ? __pfx_kthread+0x10/0x10 <4> [519.000119] ret_from_fork+0x3d4/0x480 <4> [519.000122] ? __pfx_kthread+0x10/0x10 <4> [519.000126] ret_from_fork_asm+0x1a/0x30 <4> [519.000133] </TASK> <4> [519.000135] irq event stamp: 215307 <4> [519.000136] hardirqs last enabled at (215313): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.000139] hardirqs last disabled at (215318): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.000142] softirqs last enabled at (213994): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.000144] softirqs last disabled at (213989): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.000147] ---[ end trace 0000000000000000 ]--- <6> [519.000149] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.000219] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.000226] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.000642] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.000887] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.000977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.001071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.001160] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.001249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.001336] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.001431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.001521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.001604] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.001701] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.001819] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.002827] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.013542] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.013791] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.014857] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.014937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.015016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.015094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.015170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.015245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.015319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.015405] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.015480] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.015555] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.015631] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.015705] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.015780] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.015858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.015934] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.016009] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.016085] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.016160] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.016234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.016314] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.016405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.016485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.016566] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.016644] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.016727] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.016810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.016890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.016972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.017053] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.017132] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.017212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.017292] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.017379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.017468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.017556] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.017640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.017722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.017801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.017879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.017957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.018037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.018117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.018200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.018281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.018368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.018450] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.018528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.018610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.018693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.018772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.018851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.018932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.019011] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.019089] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.019166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.019243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.019318] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.019402] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.019482] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.019737] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.019741] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=3, flags=0x4 in no process [-1] <7> [519.019744] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.019803] ------------[ cut here ]------------ <4> [519.019804] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.019805] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#11: kworker/u64:17/5666 <4> [519.019878] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.019953] autofs4 [last unloaded: snd_hda_intel] <4> [519.019957] CPU: 11 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.019960] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.019962] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.019963] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.019968] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.020039] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.020040] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [519.020043] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.020044] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.020045] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.020047] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.020048] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.020049] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000 <4> [519.020051] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.020052] CR2: 000071e234f6d000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.020054] PKRU: 55555554 <4> [519.020055] Call Trace: <4> [519.020056] <TASK> <4> [519.020060] ? lock_sync+0x100/0x100 <4> [519.020065] ? lock_release+0xd0/0x2b0 <4> [519.020070] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.020076] process_one_work+0x239/0x760 <4> [519.020082] worker_thread+0x200/0x3f0 <4> [519.020084] ? __pfx_worker_thread+0x10/0x10 <4> [519.020087] kthread+0x10d/0x150 <4> [519.020090] ? __pfx_kthread+0x10/0x10 <4> [519.020093] ret_from_fork+0x3d4/0x480 <4> [519.020095] ? __pfx_kthread+0x10/0x10 <4> [519.020099] ret_from_fork_asm+0x1a/0x30 <4> [519.020106] </TASK> <4> [519.020107] irq event stamp: 219333 <4> [519.020108] hardirqs last enabled at (219339): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.020111] hardirqs last disabled at (219344): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.020113] softirqs last enabled at (218116): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.020116] softirqs last disabled at (218111): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.020118] ---[ end trace 0000000000000000 ]--- <7> [519.021153] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.021265] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.028252] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.028466] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.028594] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.028716] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.028832] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.028950] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.029266] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.029393] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.029511] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.029627] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.029743] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.029860] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.030005] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.030124] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.030240] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.030354] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.030487] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.030601] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.030713] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.030826] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.030938] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.031051] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.031163] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.031275] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.038311] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.038463] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.038570] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.038675] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.038778] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.038883] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.038990] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.039097] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.039203] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.039309] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.039440] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.039557] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.039693] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.039808] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.039921] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.040034] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.040146] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.040259] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.040381] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.040495] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.040608] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.040719] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.040832] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.040943] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.083110] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=1, flags=0x4 in no process [-1] <7> [519.083138] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.083694] ------------[ cut here ]------------ <4> [519.083699] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.083703] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#2: kworker/u64:4/203 <4> [519.084292] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.085027] autofs4 [last unloaded: snd_hda_intel] <4> [519.085064] CPU: 2 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.085091] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.085103] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.085116] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.085159] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.085797] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.085814] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [519.085838] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.085852] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.085865] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.085877] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.085889] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.085901] FS: 0000000000000000(0000) GS:ffff8888dad97000(0000) knlGS:0000000000000000 <4> [519.085917] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.085930] CR2: 000000c0004cb000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.085944] PKRU: 55555554 <4> [519.085956] Call Trace: <4> [519.085967] <TASK> <4> [519.086007] ? lock_sync+0x100/0x100 <4> [519.086051] ? lock_release+0xd0/0x2b0 <4> [519.086103] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.086155] process_one_work+0x239/0x760 <4> [519.086215] worker_thread+0x200/0x3f0 <4> [519.086241] ? __pfx_worker_thread+0x10/0x10 <4> [519.086263] kthread+0x10d/0x150 <4> [519.086286] ? __pfx_kthread+0x10/0x10 <4> [519.086319] ret_from_fork+0x3d4/0x480 <4> [519.086337] ? __pfx_kthread+0x10/0x10 <4> [519.086389] ret_from_fork_asm+0x1a/0x30 <4> [519.086462] </TASK> <4> [519.086474] irq event stamp: 681933 <4> [519.086485] hardirqs last enabled at (681939): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.086509] hardirqs last disabled at (681944): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.086528] softirqs last enabled at (681234): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.086549] softirqs last disabled at (681229): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.086570] ---[ end trace 0000000000000000 ]--- <6> [519.086588] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.087407] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.087451] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.087888] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.089394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.089898] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.090408] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.090874] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.091328] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.091839] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.092288] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.092764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.093178] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.093648] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.094072] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.095568] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.106601] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.106885] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.108019] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.108125] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.108226] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.108327] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.108438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.108535] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.108634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.108734] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.108833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.108932] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.109031] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.109129] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.109227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.109321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.109423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.109518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.109608] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.109696] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.109787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.109884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.109981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.110074] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.110165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.110252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.110334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.110423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.110505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.110593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.110681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.110766] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.110851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.110936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.111027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.111116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.111206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.111291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.111379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.111461] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.111547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.111633] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.111721] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.111806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.111888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.111969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.112054] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.112136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.112218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.112304] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.112397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.112479] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.112562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.112646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.112731] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.112818] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.112902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.112987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.113069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.113151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.113235] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.113491] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.113496] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=1, flags=0x4 in no process [-1] <7> [519.113498] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.113561] ------------[ cut here ]------------ <4> [519.113563] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.113564] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:4/203 <4> [519.113640] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.113707] autofs4 [last unloaded: snd_hda_intel] <4> [519.113712] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.113715] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.113716] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.113717] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.113723] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.113796] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.113798] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [519.113800] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.113802] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.113803] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.113804] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.113805] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.113807] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [519.113809] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.113810] CR2: 0000799f62a14000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.113812] PKRU: 55555554 <4> [519.113813] Call Trace: <4> [519.113814] <TASK> <4> [519.113818] ? lock_sync+0x100/0x100 <4> [519.113823] ? lock_release+0xd0/0x2b0 <4> [519.113828] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.113834] process_one_work+0x239/0x760 <4> [519.113841] worker_thread+0x200/0x3f0 <4> [519.113844] ? __pfx_worker_thread+0x10/0x10 <4> [519.113846] kthread+0x10d/0x150 <4> [519.113849] ? __pfx_kthread+0x10/0x10 <4> [519.113853] ret_from_fork+0x3d4/0x480 <4> [519.113855] ? __pfx_kthread+0x10/0x10 <4> [519.113858] ret_from_fork_asm+0x1a/0x30 <4> [519.113866] </TASK> <4> [519.113867] irq event stamp: 685995 <4> [519.113868] hardirqs last enabled at (686001): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.113871] hardirqs last disabled at (686006): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.113874] softirqs last enabled at (684720): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.113876] softirqs last disabled at (684713): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.113879] ---[ end trace 0000000000000000 ]--- <6> [519.113881] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.113954] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.113960] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.114496] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.114745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.114838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.114936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.115028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.115119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.115209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.115299] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.115397] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.115485] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.115587] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.115708] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.116721] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.127701] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.127948] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.129015] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.129093] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.129169] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.129245] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.129318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.129403] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.129483] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.129562] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.129642] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.129722] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.129804] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.129886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.129970] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.130051] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.130132] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.130212] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.130292] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.130374] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.130454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.130540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.130627] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.130711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.130798] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.130884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.130967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.131049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.131131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.131214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.131298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.131386] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.131473] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.131562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.131654] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.131746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.131836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.131923] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.132008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.132090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.132172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.132253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.132338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.132430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.132517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.132602] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.132690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.132774] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.132858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.132939] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.133023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.133105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.133192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.133280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.133369] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.133455] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.133537] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.133622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.133704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.133785] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.133870] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.134125] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.134129] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=1, flags=0x4 in no process [-1] <7> [519.134131] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.134192] ------------[ cut here ]------------ <4> [519.134194] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.134195] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:4/203 <4> [519.134271] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.134336] autofs4 [last unloaded: snd_hda_intel] <4> [519.134339] CPU: 6 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.134342] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.134344] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.134345] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.134350] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.134437] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.134439] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [519.134441] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.134443] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.134444] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.134446] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.134447] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.134449] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [519.134450] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.134452] CR2: 0000799f62a14000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.134454] PKRU: 55555554 <4> [519.134455] Call Trace: <4> [519.134456] <TASK> <4> [519.134460] ? lock_sync+0x100/0x100 <4> [519.134464] ? lock_release+0xd0/0x2b0 <4> [519.134470] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.134476] process_one_work+0x239/0x760 <4> [519.134482] worker_thread+0x200/0x3f0 <4> [519.134485] ? __pfx_worker_thread+0x10/0x10 <4> [519.134488] kthread+0x10d/0x150 <4> [519.134491] ? __pfx_kthread+0x10/0x10 <4> [519.134495] ret_from_fork+0x3d4/0x480 <4> [519.134497] ? __pfx_kthread+0x10/0x10 <4> [519.134501] ret_from_fork_asm+0x1a/0x30 <4> [519.134508] </TASK> <4> [519.134510] irq event stamp: 689995 <4> [519.134511] hardirqs last enabled at (690001): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.134514] hardirqs last disabled at (690006): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.134517] softirqs last enabled at (688476): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.134520] softirqs last disabled at (688467): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.134522] ---[ end trace 0000000000000000 ]--- <7> [519.135539] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.135654] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.142605] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.142835] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.142959] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.143079] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.143198] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.143314] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.143449] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.143568] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.143688] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.143807] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.143923] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.144039] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.144179] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.144297] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.144422] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.144535] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.144648] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.144915] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.145028] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.145140] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.145255] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.145373] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.145487] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.145602] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.152707] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.152860] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.152977] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.153097] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.153215] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.153331] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.153463] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.153579] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.153694] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.153810] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.153925] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.154040] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.154177] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.154293] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.154416] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.154533] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.154647] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.154759] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.154872] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.154987] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.155101] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.155213] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.155325] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.155454] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.198390] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=3, flags=0x4 in no process [-1] <7> [519.198415] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.198777] ------------[ cut here ]------------ <4> [519.198785] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.198793] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#5: kworker/u64:5/204 <4> [519.199203] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.199588] autofs4 [last unloaded: snd_hda_intel] <4> [519.199610] CPU: 5 UID: 0 PID: 204 Comm: kworker/u64:5 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.199626] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.199632] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.199639] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.199667] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.200060] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.200070] RSP: 0018:ffffc90001627ca0 EFLAGS: 00010246 <4> [519.200083] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.200093] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.200101] RBP: ffffc90001627db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.200108] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.200115] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.200123] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000 <4> [519.200132] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.200140] CR2: 000059da42793190 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.200149] PKRU: 55555554 <4> [519.200156] Call Trace: <4> [519.200163] <TASK> <4> [519.200186] ? lock_sync+0x100/0x100 <4> [519.200211] ? lock_release+0xd0/0x2b0 <4> [519.200241] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.200271] process_one_work+0x239/0x760 <4> [519.200305] worker_thread+0x200/0x3f0 <4> [519.200319] ? __pfx_worker_thread+0x10/0x10 <4> [519.200333] kthread+0x10d/0x150 <4> [519.200349] ? __pfx_kthread+0x10/0x10 <4> [519.200389] ret_from_fork+0x3d4/0x480 <4> [519.200400] ? __pfx_kthread+0x10/0x10 <4> [519.200418] ret_from_fork_asm+0x1a/0x30 <4> [519.200458] </TASK> <4> [519.200465] irq event stamp: 536923 <4> [519.200472] hardirqs last enabled at (536929): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.200489] hardirqs last disabled at (536934): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.200502] softirqs last enabled at (536894): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.200516] softirqs last disabled at (536889): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.200528] ---[ end trace 0000000000000000 ]--- <6> [519.200538] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.200919] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.200952] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.201399] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.202270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.202820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.203325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.204149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.204891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.205527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.206168] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.206716] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.206802] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.206899] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.207019] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.208069] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.218723] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.219035] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.220312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.220502] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.220636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.220767] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.220898] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.221026] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.221158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.221297] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.221464] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.221675] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.221814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.221951] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.222089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.222225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.222368] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.222500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.222625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.222751] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.222876] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.223020] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.223163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.223294] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.223438] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.223588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.223711] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.223828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.223944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.224066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.224188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.224302] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.224437] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.224563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.224680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.224793] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.224903] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.225010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.225118] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.225226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.225330] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.225442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.225560] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.225662] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.225762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.225856] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.225955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.226050] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.226144] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.226237] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.226326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.226425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.226521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.226609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.226692] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.226776] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.226857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.226944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.227026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.227108] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.227192] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.227449] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.227454] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=3, flags=0x4 in no process [-1] <7> [519.227456] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.227515] ------------[ cut here ]------------ <4> [519.227517] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.227518] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#11: kworker/u64:8/990 <4> [519.227590] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.227665] autofs4 [last unloaded: snd_hda_intel] <4> [519.227669] CPU: 11 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.227672] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.227674] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.227675] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.227680] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.227749] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.227751] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [519.227753] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.227755] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.227756] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.227757] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.227759] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.227760] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000 <4> [519.227762] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.227763] CR2: 000071e234f6d000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.227765] PKRU: 55555554 <4> [519.227766] Call Trace: <4> [519.227767] <TASK> <4> [519.227771] ? lock_sync+0x100/0x100 <4> [519.227776] ? lock_release+0xd0/0x2b0 <4> [519.227782] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.227787] process_one_work+0x239/0x760 <4> [519.227794] worker_thread+0x200/0x3f0 <4> [519.227797] ? __pfx_worker_thread+0x10/0x10 <4> [519.227799] kthread+0x10d/0x150 <4> [519.227802] ? __pfx_kthread+0x10/0x10 <4> [519.227806] ret_from_fork+0x3d4/0x480 <4> [519.227808] ? __pfx_kthread+0x10/0x10 <4> [519.227811] ret_from_fork_asm+0x1a/0x30 <4> [519.227819] </TASK> <4> [519.227820] irq event stamp: 352133 <4> [519.227821] hardirqs last enabled at (352139): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.227824] hardirqs last disabled at (352144): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.227826] softirqs last enabled at (350802): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.227829] softirqs last disabled at (350795): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.227831] ---[ end trace 0000000000000000 ]--- <6> [519.227833] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.227902] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.227909] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.228315] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.228773] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.228867] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.228963] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.229051] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.229139] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.229226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.229313] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.229421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.229511] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.229609] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.229725] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.230766] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.241497] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.241740] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.242784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.242864] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.242936] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.243010] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.243081] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.243152] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.243222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.243291] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.243366] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.243440] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.243515] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.243589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.243665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.243739] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.243814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.243890] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.243966] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.244043] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.244117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.244198] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.244279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.244364] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.244444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.244523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.244605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.244684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.244763] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.244844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.244926] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.245005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.245085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.245165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.245253] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.245340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.245436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.245519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.245600] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.245677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.245756] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.245832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.245911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.245989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.246066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.246143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.246224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.246301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.246381] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.246457] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.246539] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.246618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.246697] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.246778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.246855] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.246932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.247008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.247085] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.247161] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.247236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.247314] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.247575] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.247579] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=3, flags=0x4 in no process [-1] <7> [519.247581] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.247640] ------------[ cut here ]------------ <4> [519.247641] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.247643] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#11: kworker/u64:8/990 <4> [519.247714] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.247779] autofs4 [last unloaded: snd_hda_intel] <4> [519.247783] CPU: 11 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.247786] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.247787] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.247789] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.247795] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.247864] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.247866] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [519.247868] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.247870] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.247872] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.247873] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.247874] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.247876] FS: 0000000000000000(0000) GS:ffff8888db217000(0000) knlGS:0000000000000000 <4> [519.247878] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.247879] CR2: 000071e234f6d000 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.247881] PKRU: 55555554 <4> [519.247883] Call Trace: <4> [519.247884] <TASK> <4> [519.247888] ? lock_sync+0x100/0x100 <4> [519.247893] ? lock_release+0xd0/0x2b0 <4> [519.247898] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.247904] process_one_work+0x239/0x760 <4> [519.247911] worker_thread+0x200/0x3f0 <4> [519.247914] ? __pfx_worker_thread+0x10/0x10 <4> [519.247916] kthread+0x10d/0x150 <4> [519.247919] ? __pfx_kthread+0x10/0x10 <4> [519.247923] ret_from_fork+0x3d4/0x480 <4> [519.247925] ? __pfx_kthread+0x10/0x10 <4> [519.247929] ret_from_fork_asm+0x1a/0x30 <4> [519.247936] </TASK> <4> [519.247938] irq event stamp: 356161 <4> [519.247939] hardirqs last enabled at (356167): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.247942] hardirqs last disabled at (356172): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.247944] softirqs last enabled at (354960): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.247946] softirqs last disabled at (354953): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.247949] ---[ end trace 0000000000000000 ]--- <7> [519.249006] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.249127] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.255532] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.255804] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.255927] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.256046] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.256163] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.256278] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.256404] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.256520] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.256635] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.256751] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.256866] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.256982] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.257122] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.257242] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.257364] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.257482] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.257596] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.257709] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.257822] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.257934] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.258046] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.258159] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.258272] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.258388] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.265515] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.265666] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.265783] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.265897] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.266129] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.266246] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.266367] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.266484] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.266600] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.266714] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.266829] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.266942] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.267077] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.267194] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.267306] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.267427] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.267540] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.267653] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.267766] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.267882] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.267997] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.268109] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.268222] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.268334] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.310893] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=1, flags=0x4 in no process [-1] <7> [519.310921] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.311429] ------------[ cut here ]------------ <4> [519.311438] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.311448] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#13: kworker/u64:3/194 <4> [519.311952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.312419] autofs4 [last unloaded: snd_hda_intel] <4> [519.312445] CPU: 13 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.312466] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.312474] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.312484] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.312517] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.312997] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.313009] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.313022] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.313031] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.313040] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.313047] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.313055] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.313064] FS: 0000000000000000(0000) GS:ffff8888db317000(0000) knlGS:0000000000000000 <4> [519.313073] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.313082] CR2: 0000799f5805f448 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [519.313091] PKRU: 55555554 <4> [519.313098] Call Trace: <4> [519.313105] <TASK> <4> [519.313132] ? lock_sync+0x100/0x100 <4> [519.313163] ? lock_release+0xd0/0x2b0 <4> [519.313199] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.313237] process_one_work+0x239/0x760 <4> [519.313280] worker_thread+0x200/0x3f0 <4> [519.313297] ? __pfx_worker_thread+0x10/0x10 <4> [519.313311] kthread+0x10d/0x150 <4> [519.313327] ? __pfx_kthread+0x10/0x10 <4> [519.313349] ret_from_fork+0x3d4/0x480 <4> [519.313381] ? __pfx_kthread+0x10/0x10 <4> [519.313403] ret_from_fork_asm+0x1a/0x30 <4> [519.313454] </TASK> <4> [519.313461] irq event stamp: 805893 <4> [519.313468] hardirqs last enabled at (805899): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.313485] hardirqs last disabled at (805904): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.313498] softirqs last enabled at (805126): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.313513] softirqs last disabled at (805119): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.313526] ---[ end trace 0000000000000000 ]--- <6> [519.313633] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.314092] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.314136] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.314606] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.315436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.315922] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.316420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.316879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.317325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.317830] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.318274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.318778] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.319221] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.319757] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.320279] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.321901] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.332534] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.332805] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.334060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.334146] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.334222] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.334307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.334401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.334501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.334581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.334667] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.334753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.334833] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.334916] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.334998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.335089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.335191] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.335266] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.335347] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.335446] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.335549] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.335622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.335704] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.335804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.335879] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.335961] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.336033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.336111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.336185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.336258] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.336333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.336470] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.336570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.336642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.336717] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.336801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.336881] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.336960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.337046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.337138] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.337214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.337295] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.337372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.337490] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.337565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.337636] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.337706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.337779] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.337849] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.337920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.337990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.338063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.338137] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.338209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.338281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.338352] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.338439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.338541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.338618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.338694] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.338770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.338850] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.339125] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.339130] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=1, flags=0x4 in no process [-1] <7> [519.339133] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.339192] ------------[ cut here ]------------ <4> [519.339193] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.339194] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.339273] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.339340] autofs4 [last unloaded: snd_hda_intel] <4> [519.339344] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.339347] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.339349] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.339350] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.339367] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.339475] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.339478] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.339483] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.339486] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.339487] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.339489] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.339490] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.339492] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.339494] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.339495] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.339497] PKRU: 55555554 <4> [519.339499] Call Trace: <4> [519.339500] <TASK> <4> [519.339504] ? lock_sync+0x100/0x100 <4> [519.339510] ? lock_release+0xd0/0x2b0 <4> [519.339516] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.339522] process_one_work+0x239/0x760 <4> [519.339529] worker_thread+0x200/0x3f0 <4> [519.339532] ? __pfx_worker_thread+0x10/0x10 <4> [519.339535] kthread+0x10d/0x150 <4> [519.339539] ? __pfx_kthread+0x10/0x10 <4> [519.339543] ret_from_fork+0x3d4/0x480 <4> [519.339545] ? __pfx_kthread+0x10/0x10 <4> [519.339549] ret_from_fork_asm+0x1a/0x30 <4> [519.339557] </TASK> <4> [519.339558] irq event stamp: 809943 <4> [519.339560] hardirqs last enabled at (809949): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.339563] hardirqs last disabled at (809954): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.339566] softirqs last enabled at (809772): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.339569] softirqs last disabled at (809761): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.339571] ---[ end trace 0000000000000000 ]--- <6> [519.339574] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.339646] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.339653] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.340065] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.340316] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.340425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.340547] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.340638] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.340753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.340840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.340932] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.341026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.341109] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.341206] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.341329] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.342383] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.353119] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.353365] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.354592] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.354666] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.354737] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.354810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.354882] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.354955] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.355027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.355096] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.355165] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.355234] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.355303] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.355378] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.355477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.355551] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.355625] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.355699] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.355803] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.355888] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.355967] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.356057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.356145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.356231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.356312] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.356405] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.356519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.356606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.356685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.356772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.356852] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.356936] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.357016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.357096] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.357182] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.357291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.357392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.357501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.357584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.357672] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.357753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.357835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.357919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.358023] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.358104] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.358189] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.358271] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.358354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.358448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.358545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.358632] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.358733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.358813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.358901] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.358980] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.359065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.359142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.359221] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.359298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.359409] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.359507] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.359771] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.359775] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=1, flags=0x4 in no process [-1] <7> [519.359778] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.359838] ------------[ cut here ]------------ <4> [519.359839] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.359841] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.359913] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.359978] autofs4 [last unloaded: snd_hda_intel] <4> [519.359983] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.359986] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.359987] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.359988] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.359994] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.360068] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.360071] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.360075] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.360078] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.360080] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.360083] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.360085] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.360087] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.360090] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.360092] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.360095] PKRU: 55555554 <4> [519.360097] Call Trace: <4> [519.360099] <TASK> <4> [519.360106] ? lock_sync+0x100/0x100 <4> [519.360114] ? lock_release+0xd0/0x2b0 <4> [519.360122] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.360132] process_one_work+0x239/0x760 <4> [519.360143] worker_thread+0x200/0x3f0 <4> [519.360148] ? __pfx_worker_thread+0x10/0x10 <4> [519.360153] kthread+0x10d/0x150 <4> [519.360157] ? __pfx_kthread+0x10/0x10 <4> [519.360163] ret_from_fork+0x3d4/0x480 <4> [519.360167] ? __pfx_kthread+0x10/0x10 <4> [519.360173] ret_from_fork_asm+0x1a/0x30 <4> [519.360181] </TASK> <4> [519.360183] irq event stamp: 813939 <4> [519.360184] hardirqs last enabled at (813945): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.360187] hardirqs last disabled at (813950): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.360189] softirqs last enabled at (812810): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.360192] softirqs last disabled at (812805): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.360194] ---[ end trace 0000000000000000 ]--- <7> [519.361478] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.361588] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.368803] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.369183] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.369326] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.369505] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.369642] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.369778] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.369913] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.370047] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.370179] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.370312] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.370476] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.370612] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.370776] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.370901] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.371021] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.371144] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.371263] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.371390] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.371523] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.371653] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.371783] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.371913] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.372045] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.372176] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.379667] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.379828] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.379950] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.380067] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.380184] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.380300] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.380426] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.380542] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.380658] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.380774] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.380888] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.381004] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.381141] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.381257] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.381379] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.381569] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.381682] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.381795] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.381908] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.382020] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.382135] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.382252] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.382373] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.382487] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.423263] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=3, flags=0x4 in no process [-1] <7> [519.423287] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.423794] ------------[ cut here ]------------ <4> [519.423803] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.423811] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.424193] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.424598] autofs4 [last unloaded: snd_hda_intel] <4> [519.424623] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.424639] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.424646] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.424655] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.424681] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.425042] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.425052] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.425064] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.425072] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.425078] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.425085] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.425091] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.425099] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.425107] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.425114] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.425122] PKRU: 55555554 <4> [519.425128] Call Trace: <4> [519.425134] <TASK> <4> [519.425155] ? lock_sync+0x100/0x100 <4> [519.425178] ? lock_release+0xd0/0x2b0 <4> [519.425207] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.425235] process_one_work+0x239/0x760 <4> [519.425267] worker_thread+0x200/0x3f0 <4> [519.425281] ? __pfx_worker_thread+0x10/0x10 <4> [519.425294] kthread+0x10d/0x150 <4> [519.425308] ? __pfx_kthread+0x10/0x10 <4> [519.425328] ret_from_fork+0x3d4/0x480 <4> [519.425338] ? __pfx_kthread+0x10/0x10 <4> [519.425377] ret_from_fork_asm+0x1a/0x30 <4> [519.425418] </TASK> <4> [519.425425] irq event stamp: 814993 <4> [519.425432] hardirqs last enabled at (814999): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.425448] hardirqs last disabled at (815004): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.425460] softirqs last enabled at (814228): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.425473] softirqs last disabled at (814221): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.425485] ---[ end trace 0000000000000000 ]--- <6> [519.425495] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.425870] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.425924] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.426401] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.429264] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.429834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.430417] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.430860] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.431272] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.431696] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.432077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.432481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.432824] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.433222] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.433671] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.435123] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.445368] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.445639] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.446718] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.446814] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.446906] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.446998] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.447089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.447179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.447262] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.447344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.447443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.447524] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.447606] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.447686] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.447762] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.447837] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.447911] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.447987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.448060] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.448135] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.448209] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.448293] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.448410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.448505] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.448588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.448669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.448747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.448826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.448904] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.448985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.449066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.449145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.449223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.449301] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.449393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.449481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.449571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.449657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.449739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.449820] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.449899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.449979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.450064] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.450145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.450225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.450303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.450401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.450485] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.450563] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.450642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.450726] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.450806] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.450886] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.450968] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.451046] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.451124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.451201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.451279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.451354] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.451445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.451524] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.451797] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.451804] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=3, flags=0x4 in no process [-1] <7> [519.451807] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.451866] ------------[ cut here ]------------ <4> [519.451867] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.451869] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#8: kworker/u64:14/5663 <4> [519.451941] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.452010] autofs4 [last unloaded: snd_hda_intel] <4> [519.452014] CPU: 8 UID: 0 PID: 5663 Comm: kworker/u64:14 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.452017] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.452019] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.452021] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.452026] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.452097] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.452099] RSP: 0018:ffffc90003473ca0 EFLAGS: 00010246 <4> [519.452102] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.452103] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.452105] RBP: ffffc90003473db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.452106] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.452108] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.452109] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [519.452111] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.452113] CR2: 0000799f580621e8 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [519.452115] PKRU: 55555554 <4> [519.452116] Call Trace: <4> [519.452117] <TASK> <4> [519.452122] ? lock_sync+0x100/0x100 <4> [519.452127] ? lock_release+0xd0/0x2b0 <4> [519.452132] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.452138] process_one_work+0x239/0x760 <4> [519.452145] worker_thread+0x200/0x3f0 <4> [519.452148] ? __pfx_worker_thread+0x10/0x10 <4> [519.452151] kthread+0x10d/0x150 <4> [519.452154] ? __pfx_kthread+0x10/0x10 <4> [519.452158] ret_from_fork+0x3d4/0x480 <4> [519.452160] ? __pfx_kthread+0x10/0x10 <4> [519.452164] ret_from_fork_asm+0x1a/0x30 <4> [519.452171] </TASK> <4> [519.452173] irq event stamp: 178141 <4> [519.452174] hardirqs last enabled at (178147): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.452177] hardirqs last disabled at (178152): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.452179] softirqs last enabled at (176828): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.452181] softirqs last disabled at (176821): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.452184] ---[ end trace 0000000000000000 ]--- <6> [519.452186] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.452258] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.452266] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.453529] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.453635] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.453742] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.453990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.454080] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.454172] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.454274] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.454372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.454462] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.454550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.454640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.454724] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.454821] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.454938] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.455948] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.466365] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.466614] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.467639] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.467719] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.467796] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.467875] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.467953] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.468030] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.468107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.468184] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.468260] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.468335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.468423] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.468501] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.468581] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.468659] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.468735] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.468810] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.468885] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.468960] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.469035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.469117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.469199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.469279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.469365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.469445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.469527] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.469635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.469714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.469794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.469875] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.469955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.470035] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.470114] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.470199] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.470284] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.470377] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.470460] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.470541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.470622] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.470706] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.470787] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.470869] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.470949] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.471028] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.471105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.471187] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.471265] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.471343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.471432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.471525] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.471606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.471686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.471767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.471848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.471928] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.472004] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.472084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.472159] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.472236] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.472316] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.472609] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.472614] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=3, flags=0x4 in no process [-1] <7> [519.472617] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.472678] ------------[ cut here ]------------ <4> [519.472679] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.472681] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.472752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.472818] autofs4 [last unloaded: snd_hda_intel] <4> [519.472822] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.472825] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.472826] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.472828] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.472833] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.472903] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.472904] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.472907] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.472908] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.472910] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.472911] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.472912] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.472913] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.472915] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.472916] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.472918] PKRU: 55555554 <4> [519.472919] Call Trace: <4> [519.472920] <TASK> <4> [519.472924] ? lock_sync+0x100/0x100 <4> [519.472929] ? lock_release+0xd0/0x2b0 <4> [519.472934] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.472940] process_one_work+0x239/0x760 <4> [519.472946] worker_thread+0x200/0x3f0 <4> [519.472949] ? __pfx_worker_thread+0x10/0x10 <4> [519.472952] kthread+0x10d/0x150 <4> [519.472954] ? __pfx_kthread+0x10/0x10 <4> [519.472958] ret_from_fork+0x3d4/0x480 <4> [519.472960] ? __pfx_kthread+0x10/0x10 <4> [519.472964] ret_from_fork_asm+0x1a/0x30 <4> [519.472971] </TASK> <4> [519.472972] irq event stamp: 819667 <4> [519.472973] hardirqs last enabled at (819673): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.472976] hardirqs last disabled at (819678): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.472979] softirqs last enabled at (818284): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.472981] softirqs last disabled at (818277): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.472984] ---[ end trace 0000000000000000 ]--- <7> [519.474087] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.474203] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.480882] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.481251] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.481384] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.481505] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.481625] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.481743] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.481859] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.481975] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.482090] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.482209] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.482326] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.482451] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.482594] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.482711] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.482826] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.482939] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.483052] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.483165] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.483277] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.483401] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.483515] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.483627] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.483738] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.483851] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.490931] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.491090] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.491210] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.491327] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.491453] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.491568] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.491685] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.491800] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.491914] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.492029] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.492144] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.492258] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.492406] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.492524] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.492637] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.492753] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.492868] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.492982] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.493096] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.493208] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.493319] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.493442] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.493555] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.493667] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.536043] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=1, flags=0x4 in no process [-1] <7> [519.536059] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.536424] ------------[ cut here ]------------ <4> [519.536431] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.536440] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.536822] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.537214] autofs4 [last unloaded: snd_hda_intel] <4> [519.537236] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.537252] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.537259] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.537267] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.537294] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.537729] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.537740] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.537752] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.537760] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.537767] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.537773] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.537779] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.537786] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.537795] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.537801] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.537810] PKRU: 55555554 <4> [519.537816] Call Trace: <4> [519.537823] <TASK> <4> [519.537844] ? lock_sync+0x100/0x100 <4> [519.537867] ? lock_release+0xd0/0x2b0 <4> [519.537895] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.537926] process_one_work+0x239/0x760 <4> [519.537959] worker_thread+0x200/0x3f0 <4> [519.537973] ? __pfx_worker_thread+0x10/0x10 <4> [519.537986] kthread+0x10d/0x150 <4> [519.538002] ? __pfx_kthread+0x10/0x10 <4> [519.538021] ret_from_fork+0x3d4/0x480 <4> [519.538032] ? __pfx_kthread+0x10/0x10 <4> [519.538050] ret_from_fork_asm+0x1a/0x30 <4> [519.538089] </TASK> <4> [519.538095] irq event stamp: 822399 <4> [519.538101] hardirqs last enabled at (822405): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.538116] hardirqs last disabled at (822410): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.538127] softirqs last enabled at (821636): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.538141] softirqs last disabled at (821631): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.538153] ---[ end trace 0000000000000000 ]--- <6> [519.538163] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.538593] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.538625] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.539060] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.539840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.540285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.540796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.541226] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.541685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.542094] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.542545] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.542969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.543440] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.543889] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.544325] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.545948] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.556766] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.557032] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.558264] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.558345] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.558438] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.558518] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.558598] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.558677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.558758] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.558839] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.558918] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.558999] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.559079] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.559159] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.559239] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.559316] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.559401] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.559477] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.559552] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.559626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.559700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.559781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.559862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.559942] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.560022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.560099] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.560177] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.560256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.560334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.560431] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.560513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.560592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.560673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.560751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.560835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.560920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.561003] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.561084] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.561164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.561242] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.561323] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.561412] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.561493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.561571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.561648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.561724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.561805] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.561883] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.561960] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.562037] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.562115] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.562191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.562267] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.562346] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.562436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.562513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.562592] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.562674] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.562751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.562828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.562908] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.563170] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.563175] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=1, flags=0x4 in no process [-1] <7> [519.563178] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.563237] ------------[ cut here ]------------ <4> [519.563238] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.563239] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.563310] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.563386] autofs4 [last unloaded: snd_hda_intel] <4> [519.563390] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.563393] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.563394] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.563396] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.563402] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.563471] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.563473] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.563476] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.563477] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.563479] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.563480] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.563482] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.563483] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.563485] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.563487] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.563488] PKRU: 55555554 <4> [519.563489] Call Trace: <4> [519.563491] <TASK> <4> [519.563494] ? lock_sync+0x100/0x100 <4> [519.563499] ? lock_release+0xd0/0x2b0 <4> [519.563504] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.563510] process_one_work+0x239/0x760 <4> [519.563516] worker_thread+0x200/0x3f0 <4> [519.563519] ? __pfx_worker_thread+0x10/0x10 <4> [519.563522] kthread+0x10d/0x150 <4> [519.563525] ? __pfx_kthread+0x10/0x10 <4> [519.563528] ret_from_fork+0x3d4/0x480 <4> [519.563531] ? __pfx_kthread+0x10/0x10 <4> [519.563534] ret_from_fork_asm+0x1a/0x30 <4> [519.563541] </TASK> <4> [519.563543] irq event stamp: 826451 <4> [519.563544] hardirqs last enabled at (826457): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.563547] hardirqs last disabled at (826462): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.563549] softirqs last enabled at (825198): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.563552] softirqs last disabled at (825191): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.563554] ---[ end trace 0000000000000000 ]--- <6> [519.563556] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.563625] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.563631] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.564034] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.564276] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.564368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.564464] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.564554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.564640] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.564724] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.564810] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.564897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.564980] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.565074] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.565192] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.566204] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.576932] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.577183] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.578179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.578258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.578335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.578427] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.578509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.578586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.578662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.578738] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.578813] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.578889] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.578965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.579042] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.579119] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.579194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.579269] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.579344] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.579430] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.579505] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.579580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.579663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.579745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.579827] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.579910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.579989] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.580066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.580142] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.580222] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.580305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.580395] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.580475] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.580555] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.580635] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.580720] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.580804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.580888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.580969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.581048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.581125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.581203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.581279] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.581363] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.581441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.581519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.581594] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.581675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.581755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.581838] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.581918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.581997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.582075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.582152] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.582231] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.582309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.582406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.582486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.582565] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.582642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.582719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.582800] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.583054] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.583058] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967175, lrc_seqno=4294967175, guc_id=1, flags=0x4 in no process [-1] <7> [519.583061] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.583119] ------------[ cut here ]------------ <4> [519.583121] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.583122] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.583194] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.583259] autofs4 [last unloaded: snd_hda_intel] <4> [519.583263] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.583266] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.583267] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.583268] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.583273] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.583344] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.583346] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.583348] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.583349] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.583351] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.583352] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.583353] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.583354] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.583359] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.583361] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.583362] PKRU: 55555554 <4> [519.583364] Call Trace: <4> [519.583365] <TASK> <4> [519.583369] ? lock_sync+0x100/0x100 <4> [519.583373] ? lock_release+0xd0/0x2b0 <4> [519.583379] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.583385] process_one_work+0x239/0x760 <4> [519.583391] worker_thread+0x200/0x3f0 <4> [519.583394] ? __pfx_worker_thread+0x10/0x10 <4> [519.583397] kthread+0x10d/0x150 <4> [519.583399] ? __pfx_kthread+0x10/0x10 <4> [519.583403] ret_from_fork+0x3d4/0x480 <4> [519.583405] ? __pfx_kthread+0x10/0x10 <4> [519.583409] ret_from_fork_asm+0x1a/0x30 <4> [519.583416] </TASK> <4> [519.583417] irq event stamp: 830477 <4> [519.583419] hardirqs last enabled at (830483): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.583421] hardirqs last disabled at (830488): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.583424] softirqs last enabled at (829322): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.583426] softirqs last disabled at (829311): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.583429] ---[ end trace 0000000000000000 ]--- <7> [519.584457] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.584596] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.591429] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.591644] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.591991] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.592117] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.592238] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.592366] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.592526] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.592663] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.592778] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.592895] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.593010] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.593125] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.593271] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.593426] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.593547] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.593661] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.593775] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.593889] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.594001] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.594113] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.594226] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.594339] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.594501] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.594617] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.601733] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.601894] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.602011] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.602126] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.602243] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.602369] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.602525] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.602644] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.602763] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.602881] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.602997] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.603113] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.603253] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.603383] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.603509] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.603624] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.603738] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.603851] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.603966] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.604079] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.604194] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.604309] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.604467] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.604585] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.647890] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=3, flags=0x4 in no process [-1] <7> [519.647925] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.648425] ------------[ cut here ]------------ <4> [519.648433] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.648444] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#12: kworker/u64:9/991 <4> [519.648933] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.649414] autofs4 [last unloaded: snd_hda_intel] <4> [519.649442] CPU: 12 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.649460] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.649468] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.649484] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.649516] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.649988] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.649998] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [519.650012] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.650021] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.650030] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.650038] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.650045] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.650054] FS: 0000000000000000(0000) GS:ffff8888db297000(0000) knlGS:0000000000000000 <4> [519.650063] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.650072] CR2: 000071410741f260 CR3: 000000000344c006 CR4: 0000000000f72ef0 <4> [519.650081] PKRU: 55555554 <4> [519.650088] Call Trace: <4> [519.650095] <TASK> <4> [519.650122] ? lock_sync+0x100/0x100 <4> [519.650153] ? lock_release+0xd0/0x2b0 <4> [519.650189] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.650227] process_one_work+0x239/0x760 <4> [519.650269] worker_thread+0x200/0x3f0 <4> [519.650287] ? __pfx_worker_thread+0x10/0x10 <4> [519.650301] kthread+0x10d/0x150 <4> [519.650317] ? __pfx_kthread+0x10/0x10 <4> [519.650339] ret_from_fork+0x3d4/0x480 <4> [519.650351] ? __pfx_kthread+0x10/0x10 <4> [519.650394] ret_from_fork_asm+0x1a/0x30 <4> [519.650447] </TASK> <4> [519.650456] irq event stamp: 413489 <4> [519.650464] hardirqs last enabled at (413495): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.650480] hardirqs last disabled at (413500): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.650494] softirqs last enabled at (412722): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.650508] softirqs last disabled at (412717): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.650521] ---[ end trace 0000000000000000 ]--- <6> [519.650654] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.650980] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.651024] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.651543] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.652349] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.652893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.653400] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.653857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.654308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.654828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.655297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.655828] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.656287] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.656847] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.657349] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.658989] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.670083] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.670364] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.671409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.671503] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.671589] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.671677] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.671765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.671851] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.671938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.672024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.672113] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.672207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.672299] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.672409] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.672508] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.672604] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.672701] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.672798] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.672894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.672987] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.673081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.673186] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.673291] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.673394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.673493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.673590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.673681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.673771] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.673858] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.673947] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.674036] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.674124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.674214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.674298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.674394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.674483] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.674571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.674658] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.674743] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.674826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.674912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.674996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.675081] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.675164] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.675246] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.675326] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.675419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.675502] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.675589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.675673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.675758] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.675842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.675924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.676009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.676091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.676174] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.676255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.676339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.676434] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.676520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.676608] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.677045] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.677050] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=3, flags=0x4 in no process [-1] <7> [519.677053] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.677120] ------------[ cut here ]------------ <4> [519.677122] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.677123] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:9/991 <4> [519.677203] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.677275] autofs4 [last unloaded: snd_hda_intel] <4> [519.677279] CPU: 6 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.677282] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.677284] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.677285] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.677291] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.677372] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.677374] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [519.677377] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.677378] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.677380] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.677381] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.677383] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.677384] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [519.677386] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.677387] CR2: 000073a8a7cc9aa0 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.677389] PKRU: 55555554 <4> [519.677390] Call Trace: <4> [519.677391] <TASK> <4> [519.677396] ? lock_sync+0x100/0x100 <4> [519.677401] ? lock_release+0xd0/0x2b0 <4> [519.677406] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.677412] process_one_work+0x239/0x760 <4> [519.677419] worker_thread+0x200/0x3f0 <4> [519.677422] ? __pfx_worker_thread+0x10/0x10 <4> [519.677424] kthread+0x10d/0x150 <4> [519.677428] ? __pfx_kthread+0x10/0x10 <4> [519.677431] ret_from_fork+0x3d4/0x480 <4> [519.677434] ? __pfx_kthread+0x10/0x10 <4> [519.677437] ret_from_fork_asm+0x1a/0x30 <4> [519.677445] </TASK> <4> [519.677446] irq event stamp: 417519 <4> [519.677448] hardirqs last enabled at (417525): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.677451] hardirqs last disabled at (417530): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.677453] softirqs last enabled at (416048): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.677456] softirqs last disabled at (416041): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.677458] ---[ end trace 0000000000000000 ]--- <6> [519.677460] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.677535] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.677541] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.677949] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.678211] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.678308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.678420] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.678516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.678609] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.678701] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.678791] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.678884] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.678971] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.679072] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.679194] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.680205] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.690932] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.691187] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.692225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.692310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.692402] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.692484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.692565] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.692645] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.692726] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.692805] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.692884] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.692965] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.693045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.693127] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.693207] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.693287] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.693369] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.693447] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.693525] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.693603] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.693682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.693767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.693854] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.693940] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.694027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.694111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.694196] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.694282] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.694370] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.694456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.694541] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.694624] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.694707] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.694790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.694880] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.694969] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.695060] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.695150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.695238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.695322] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.695413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.695495] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.695580] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.695664] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.695746] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.695829] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.695913] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.695995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.696079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.696165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.696251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.696334] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.696423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.696507] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.696589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.696671] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.696752] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.696834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.696915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.696997] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.697085] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.697349] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.697353] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=3, flags=0x4 in no process [-1] <7> [519.698204] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.698265] ------------[ cut here ]------------ <4> [519.698266] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.698268] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:9/991 <4> [519.698339] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.698410] autofs4 [last unloaded: snd_hda_intel] <4> [519.698414] CPU: 6 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.698417] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.698419] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.698420] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.698425] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.698495] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.698497] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [519.698499] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.698501] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.698503] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.698504] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.698506] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.698507] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [519.698509] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.698511] CR2: 000073a8a7cc9aa0 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [519.698512] PKRU: 55555554 <4> [519.698514] Call Trace: <4> [519.698515] <TASK> <4> [519.698519] ? lock_sync+0x100/0x100 <4> [519.698524] ? lock_release+0xd0/0x2b0 <4> [519.698529] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.698535] process_one_work+0x239/0x760 <4> [519.698542] worker_thread+0x200/0x3f0 <4> [519.698545] ? __pfx_worker_thread+0x10/0x10 <4> [519.698547] kthread+0x10d/0x150 <4> [519.698551] ? __pfx_kthread+0x10/0x10 <4> [519.698554] ret_from_fork+0x3d4/0x480 <4> [519.698557] ? __pfx_kthread+0x10/0x10 <4> [519.698560] ret_from_fork_asm+0x1a/0x30 <4> [519.698567] </TASK> <4> [519.698569] irq event stamp: 421539 <4> [519.698570] hardirqs last enabled at (421545): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.698573] hardirqs last disabled at (421550): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.698575] softirqs last enabled at (420918): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.698578] softirqs last disabled at (420913): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.698580] ---[ end trace 0000000000000000 ]--- <7> [519.698595] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.698698] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.705962] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.706181] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.706309] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.706476] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.706605] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.706722] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.706837] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.706952] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.707068] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.707183] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.707296] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.707468] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.707611] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.707895] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.708010] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.708122] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.708235] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.708348] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.708511] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.708630] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.708744] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.708858] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.708971] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.709085] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.716194] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.716353] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.716524] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.716672] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.716791] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.716908] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.717025] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.717140] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.717256] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.717390] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.717515] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.717629] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.717764] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.717881] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.718015] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.718131] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.718246] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.718368] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.718501] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.718616] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.718729] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.718841] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.718953] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.719066] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.761879] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=1, flags=0x4 in no process [-1] <7> [519.761909] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.762233] ------------[ cut here ]------------ <4> [519.762240] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.762248] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#0: kworker/u64:8/990 <4> [519.762722] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.763094] autofs4 [last unloaded: snd_hda_intel] <4> [519.763118] CPU: 0 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.763134] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.763141] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.763149] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.763177] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.763630] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.763641] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [519.763654] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.763663] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.763671] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.763678] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.763685] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.763693] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [519.763702] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.763711] CR2: 000073a8a7cc9980 CR3: 0000000123a85004 CR4: 0000000000f72ef0 <4> [519.763720] PKRU: 55555554 <4> [519.763727] Call Trace: <4> [519.763735] <TASK> <4> [519.763756] ? lock_sync+0x100/0x100 <4> [519.763781] ? lock_release+0xd0/0x2b0 <4> [519.763811] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.763840] process_one_work+0x239/0x760 <4> [519.763872] worker_thread+0x200/0x3f0 <4> [519.763887] ? __pfx_worker_thread+0x10/0x10 <4> [519.763900] kthread+0x10d/0x150 <4> [519.763915] ? __pfx_kthread+0x10/0x10 <4> [519.763933] ret_from_fork+0x3d4/0x480 <4> [519.763944] ? __pfx_kthread+0x10/0x10 <4> [519.763962] ret_from_fork_asm+0x1a/0x30 <4> [519.764001] </TASK> <4> [519.764008] irq event stamp: 361825 <4> [519.764014] hardirqs last enabled at (361831): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.764029] hardirqs last disabled at (361836): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.764041] softirqs last enabled at (361080): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.764054] softirqs last disabled at (361073): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.764065] ---[ end trace 0000000000000000 ]--- <6> [519.764075] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.764491] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.764528] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.765006] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.765826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.766277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.766822] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.767266] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.767759] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.768181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.768653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.769102] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.769539] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.769963] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.770418] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.771929] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.782363] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.782631] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.783708] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.783789] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.783868] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.783946] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.784024] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.784100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.784179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.784258] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.784335] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.784424] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.784500] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.784575] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.784654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.784732] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.784809] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.784886] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.784961] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.785036] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.785112] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.785195] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.785280] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.785365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.785447] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.785526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.785603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.785680] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.785757] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.785836] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.785915] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.785993] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.786071] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.786150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.786234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.786319] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.786411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.786492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.786575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.786657] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.786737] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.786813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.786893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.786972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.787049] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.787124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.787204] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.787281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.787362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.787442] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.787524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.787603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.787682] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.787762] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.787841] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.787918] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.787994] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.788073] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.788153] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.788232] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.788313] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.788559] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.788564] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=1, flags=0x4 in no process [-1] <7> [519.788567] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.788626] ------------[ cut here ]------------ <4> [519.788627] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.788629] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#0: kworker/u64:8/990 <4> [519.788701] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.788767] autofs4 [last unloaded: snd_hda_intel] <4> [519.788771] CPU: 0 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.788774] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.788775] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.788777] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.788782] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.788852] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.788854] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [519.788856] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.788858] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.788859] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.788860] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.788862] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.788863] FS: 0000000000000000(0000) GS:ffff8888dac97000(0000) knlGS:0000000000000000 <4> [519.788865] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.788866] CR2: 000073a8a7cc9980 CR3: 000000000344c005 CR4: 0000000000f72ef0 <4> [519.788867] PKRU: 55555554 <4> [519.788869] Call Trace: <4> [519.788870] <TASK> <4> [519.788873] ? lock_sync+0x100/0x100 <4> [519.788878] ? lock_release+0xd0/0x2b0 <4> [519.788883] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.788889] process_one_work+0x239/0x760 <4> [519.788895] worker_thread+0x200/0x3f0 <4> [519.788898] ? __pfx_worker_thread+0x10/0x10 <4> [519.788900] kthread+0x10d/0x150 <4> [519.788903] ? __pfx_kthread+0x10/0x10 <4> [519.788907] ret_from_fork+0x3d4/0x480 <4> [519.788909] ? __pfx_kthread+0x10/0x10 <4> [519.788912] ret_from_fork_asm+0x1a/0x30 <4> [519.788920] </TASK> <4> [519.788921] irq event stamp: 365883 <4> [519.788922] hardirqs last enabled at (365889): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.788925] hardirqs last disabled at (365894): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.788927] softirqs last enabled at (364398): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.788930] softirqs last disabled at (364389): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.788932] ---[ end trace 0000000000000000 ]--- <6> [519.788934] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.789003] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.789009] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.790268] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.790400] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.790507] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.790760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.790853] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.790946] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.791032] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.791117] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.791203] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.791289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.791391] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.791477] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.791572] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.791689] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.792702] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.803701] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.803954] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.804933] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.805012] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.805089] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.805167] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.805244] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.805321] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.805414] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.805497] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.805576] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.805654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.805731] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.805808] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.805883] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.805958] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.806033] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.806107] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.806181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.806255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.806329] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.806425] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.806524] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.806605] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.806686] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.806765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.806842] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.806919] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.806996] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.807075] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.807156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.807238] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.807320] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.807421] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.807519] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.807603] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.807687] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.807770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.807850] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.807929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.808009] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.808088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.808169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.808249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.808327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.808416] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.808513] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.808595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.808675] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.808753] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.808833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.808910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.808987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.809067] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.809146] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.809225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.809303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.809399] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.809489] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.809569] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.809651] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.809885] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.809890] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967176, lrc_seqno=4294967176, guc_id=1, flags=0x4 in no process [-1] <7> [519.809893] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.809951] ------------[ cut here ]------------ <4> [519.809952] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.809954] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:9/991 <4> [519.810026] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.810102] autofs4 [last unloaded: snd_hda_intel] <4> [519.810107] CPU: 10 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.810110] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.810112] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.810113] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.810118] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.810188] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.810190] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [519.810192] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.810193] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.810195] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.810196] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.810197] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.810198] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [519.810200] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.810201] CR2: 0000799f62a15000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.810203] PKRU: 55555554 <4> [519.810204] Call Trace: <4> [519.810205] <TASK> <4> [519.810209] ? lock_sync+0x100/0x100 <4> [519.810214] ? lock_release+0xd0/0x2b0 <4> [519.810219] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.810225] process_one_work+0x239/0x760 <4> [519.810232] worker_thread+0x200/0x3f0 <4> [519.810234] ? __pfx_worker_thread+0x10/0x10 <4> [519.810237] kthread+0x10d/0x150 <4> [519.810240] ? __pfx_kthread+0x10/0x10 <4> [519.810244] ret_from_fork+0x3d4/0x480 <4> [519.810246] ? __pfx_kthread+0x10/0x10 <4> [519.810249] ret_from_fork_asm+0x1a/0x30 <4> [519.810256] </TASK> <4> [519.810258] irq event stamp: 427585 <4> [519.810259] hardirqs last enabled at (427591): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.810262] hardirqs last disabled at (427596): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.810264] softirqs last enabled at (426444): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.810267] softirqs last disabled at (426433): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.810269] ---[ end trace 0000000000000000 ]--- <7> [519.811325] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.811451] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.817887] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.818144] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.818269] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.818397] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.818516] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.818634] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.818750] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.818867] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.818983] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.819098] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.819214] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.819330] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.819501] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.819624] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.819740] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.819855] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.819969] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.820082] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.820193] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.820306] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.820428] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.820543] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.820655] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.820767] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.827965] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.828124] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.828243] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.828375] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.828494] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.828611] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.828728] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.828842] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.828955] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.829068] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.829181] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.829294] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.829438] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.829556] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.829669] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.829782] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.829894] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.830006] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.830120] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.830236] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.830352] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.830484] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.830598] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.830712] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.873148] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=3, flags=0x4 in no process [-1] <7> [519.873175] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.873577] ------------[ cut here ]------------ <4> [519.873585] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.873593] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.874001] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.874421] autofs4 [last unloaded: snd_hda_intel] <4> [519.874448] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.874464] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.874472] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.874481] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.874509] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.874888] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.874898] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.874910] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.874918] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.874925] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.874931] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.874937] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.874944] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.874953] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.874959] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.874967] PKRU: 55555554 <4> [519.874974] Call Trace: <4> [519.874980] <TASK> <4> [519.875001] ? lock_sync+0x100/0x100 <4> [519.875026] ? lock_release+0xd0/0x2b0 <4> [519.875054] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.875083] process_one_work+0x239/0x760 <4> [519.875116] worker_thread+0x200/0x3f0 <4> [519.875130] ? __pfx_worker_thread+0x10/0x10 <4> [519.875143] kthread+0x10d/0x150 <4> [519.875157] ? __pfx_kthread+0x10/0x10 <4> [519.875176] ret_from_fork+0x3d4/0x480 <4> [519.875187] ? __pfx_kthread+0x10/0x10 <4> [519.875204] ret_from_fork_asm+0x1a/0x30 <4> [519.875242] </TASK> <4> [519.875248] irq event stamp: 834107 <4> [519.875254] hardirqs last enabled at (834113): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.875269] hardirqs last disabled at (834118): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.875281] softirqs last enabled at (833846): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.875293] softirqs last disabled at (833841): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.875305] ---[ end trace 0000000000000000 ]--- <6> [519.875315] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.875936] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.875990] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.876464] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.877256] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.877826] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.878285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.878792] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.879219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.879713] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.880155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.880659] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.881090] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.881610] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.882058] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.883661] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.894657] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.894912] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.895963] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.896053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.896139] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.896227] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.896310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.896404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.896485] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.896568] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.896650] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.896733] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.896815] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.896897] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.896980] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.897062] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.897145] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.897225] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.897301] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.897391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.897467] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.897550] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.897634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.897715] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.897796] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.897873] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.897950] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.898026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.898105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.898188] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.898270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.898350] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.898443] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.898522] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.898606] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.898690] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.898777] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.898862] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.898944] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.899022] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.899100] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.899175] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.899255] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.899338] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.899430] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.899509] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.899590] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.899669] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.899747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.899824] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.899902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.899979] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.900055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.900135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.900212] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.900289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.900368] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.900445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.900521] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.900596] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.900674] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.900921] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.900925] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=3, flags=0x4 in no process [-1] <7> [519.900928] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.900987] ------------[ cut here ]------------ <4> [519.900988] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.900990] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.901062] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.901129] autofs4 [last unloaded: snd_hda_intel] <4> [519.901133] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.901136] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.901137] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.901139] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.901144] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.901213] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.901215] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.901217] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.901219] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.901220] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.901221] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.901223] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.901224] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.901226] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.901227] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.901228] PKRU: 55555554 <4> [519.901230] Call Trace: <4> [519.901231] <TASK> <4> [519.901235] ? lock_sync+0x100/0x100 <4> [519.901239] ? lock_release+0xd0/0x2b0 <4> [519.901245] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.901250] process_one_work+0x239/0x760 <4> [519.901257] worker_thread+0x200/0x3f0 <4> [519.901260] ? __pfx_worker_thread+0x10/0x10 <4> [519.901262] kthread+0x10d/0x150 <4> [519.901265] ? __pfx_kthread+0x10/0x10 <4> [519.901269] ret_from_fork+0x3d4/0x480 <4> [519.901271] ? __pfx_kthread+0x10/0x10 <4> [519.901274] ret_from_fork_asm+0x1a/0x30 <4> [519.901282] </TASK> <4> [519.901283] irq event stamp: 838155 <4> [519.901284] hardirqs last enabled at (838161): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.901287] hardirqs last disabled at (838166): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.901290] softirqs last enabled at (836826): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.901292] softirqs last disabled at (836819): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.901295] ---[ end trace 0000000000000000 ]--- <6> [519.901296] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.901373] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.901382] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.901788] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.902033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.902124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.902217] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.902306] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.902407] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.902499] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.902588] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.902676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.902759] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.902855] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.902972] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.903978] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [519.914704] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [519.914954] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [519.915972] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [519.916050] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [519.916126] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [519.916205] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [519.916284] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [519.916365] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [519.916443] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [519.916519] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [519.916595] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [519.916670] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [519.916745] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [519.916821] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [519.916896] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [519.916971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [519.917046] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [519.917120] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [519.917194] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [519.917268] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [519.917341] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [519.917433] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [519.917517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [519.917597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [519.917677] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [519.917755] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [519.917831] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [519.917907] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [519.917984] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [519.918063] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [519.918147] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [519.918228] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [519.918309] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [519.918401] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [519.918486] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [519.918571] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [519.918655] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [519.918735] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [519.918817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [519.918899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [519.918981] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [519.919059] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [519.919140] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [519.919219] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [519.919297] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [519.919380] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [519.919465] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [519.919546] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [519.919625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [519.919703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [519.919783] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [519.919859] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [519.919935] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [519.920014] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [519.920091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [519.920167] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [519.920243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [519.920321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [519.920404] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [519.920481] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [519.920565] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [519.920810] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [519.920814] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=3, flags=0x4 in no process [-1] <7> [519.920816] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.920875] ------------[ cut here ]------------ <4> [519.920876] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.920877] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#9: kworker/u64:3/194 <4> [519.920949] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.921017] autofs4 [last unloaded: snd_hda_intel] <4> [519.921021] CPU: 9 UID: 0 PID: 194 Comm: kworker/u64:3 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.921024] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.921026] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.921027] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.921032] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.921102] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.921103] RSP: 0018:ffffc900015cfca0 EFLAGS: 00010246 <4> [519.921106] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.921107] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.921108] RBP: ffffc900015cfdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.921110] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.921111] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [519.921112] FS: 0000000000000000(0000) GS:ffff8888db117000(0000) knlGS:0000000000000000 <4> [519.921114] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.921115] CR2: 000073a8a7ccc9e0 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [519.921117] PKRU: 55555554 <4> [519.921118] Call Trace: <4> [519.921119] <TASK> <4> [519.921123] ? lock_sync+0x100/0x100 <4> [519.921127] ? lock_release+0xd0/0x2b0 <4> [519.921133] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.921138] process_one_work+0x239/0x760 <4> [519.921144] worker_thread+0x200/0x3f0 <4> [519.921147] ? __pfx_worker_thread+0x10/0x10 <4> [519.921149] kthread+0x10d/0x150 <4> [519.921152] ? __pfx_kthread+0x10/0x10 <4> [519.921156] ret_from_fork+0x3d4/0x480 <4> [519.921158] ? __pfx_kthread+0x10/0x10 <4> [519.921161] ret_from_fork_asm+0x1a/0x30 <4> [519.921169] </TASK> <4> [519.921170] irq event stamp: 842163 <4> [519.921171] hardirqs last enabled at (842169): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.921174] hardirqs last disabled at (842174): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.921176] softirqs last enabled at (840646): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.921178] softirqs last disabled at (840637): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.921181] ---[ end trace 0000000000000000 ]--- <7> [519.922226] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [519.922339] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [519.929136] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.929735] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.929866] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.929987] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.930105] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.930223] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.930340] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.930473] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.930590] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.930706] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.930820] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.930934] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.931074] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.931195] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.931312] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.931438] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.931552] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.931665] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.931778] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.931890] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.932003] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.932115] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.932227] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.932339] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.939495] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.939658] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.939774] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.939889] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.940002] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.940116] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.940228] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.940341] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.940472] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.940589] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.940704] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.940912] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [519.941048] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [519.941164] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [519.941276] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [519.941402] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [519.941518] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [519.941633] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [519.941748] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [519.941861] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [519.941975] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [519.942088] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [519.942201] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [519.942313] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [519.984132] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=1, flags=0x4 in no process [-1] <7> [519.984160] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [519.984646] ------------[ cut here ]------------ <4> [519.984655] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [519.984665] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#15: kworker/u64:9/991 <4> [519.985161] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [519.985572] autofs4 [last unloaded: snd_hda_intel] <4> [519.985599] CPU: 15 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [519.985618] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [519.985627] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [519.985638] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [519.985673] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [519.986142] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [519.986153] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [519.986167] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [519.986176] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [519.986185] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [519.986192] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [519.986200] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [519.986209] FS: 0000000000000000(0000) GS:ffff8888db417000(0000) knlGS:0000000000000000 <4> [519.986218] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [519.986227] CR2: 0000000625e66008 CR3: 000000000344c002 CR4: 0000000000f72ef0 <4> [519.986236] PKRU: 55555554 <4> [519.986243] Call Trace: <4> [519.986250] <TASK> <4> [519.986278] ? lock_sync+0x100/0x100 <4> [519.986308] ? lock_release+0xd0/0x2b0 <4> [519.986344] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [519.986409] process_one_work+0x239/0x760 <4> [519.986454] worker_thread+0x200/0x3f0 <4> [519.986473] ? __pfx_worker_thread+0x10/0x10 <4> [519.986488] kthread+0x10d/0x150 <4> [519.986504] ? __pfx_kthread+0x10/0x10 <4> [519.986527] ret_from_fork+0x3d4/0x480 <4> [519.986538] ? __pfx_kthread+0x10/0x10 <4> [519.986559] ret_from_fork_asm+0x1a/0x30 <4> [519.986611] </TASK> <4> [519.986619] irq event stamp: 429663 <4> [519.986625] hardirqs last enabled at (429669): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [519.986642] hardirqs last disabled at (429674): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [519.986655] softirqs last enabled at (429604): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.986671] softirqs last disabled at (429599): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [519.986685] ---[ end trace 0000000000000000 ]--- <6> [519.986826] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [519.987311] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [519.987397] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [519.987837] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [519.988678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [519.989163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [519.989693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [519.990151] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [519.990642] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [519.991079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [519.991554] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [519.991958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [519.992366] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [519.992766] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [519.992891] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [519.993906] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.004718] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.005080] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.006313] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.006481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.006622] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.006765] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.006901] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.007038] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.007177] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.007318] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.007484] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.007634] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.007781] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.007938] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.008090] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.008238] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.008386] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.008526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.008662] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.008797] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.008929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.009083] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.009234] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.009379] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.009518] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.009647] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.009772] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.009897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.010021] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.010145] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.010270] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.010403] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.010526] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.010648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.010770] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.010890] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.011010] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.011131] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.011247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.011361] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.011468] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.011574] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.011685] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.011794] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.011899] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.012000] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.012105] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.012205] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.012305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.012406] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.012501] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.012598] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.012693] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.012789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.012877] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.012962] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.013048] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.013136] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.013223] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.013308] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.013406] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.013654] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.013659] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=1, flags=0x4 in no process [-1] <7> [520.013661] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.013724] ------------[ cut here ]------------ <4> [520.013725] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.013727] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#6: kworker/u64:9/991 <4> [520.013803] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.013877] autofs4 [last unloaded: snd_hda_intel] <4> [520.013881] CPU: 6 UID: 0 PID: 991 Comm: kworker/u64:9 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.013885] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.013886] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.013888] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.013894] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.013969] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.013971] RSP: 0018:ffffc9000365fca0 EFLAGS: 00010246 <4> [520.013974] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.013976] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.013978] RBP: ffffc9000365fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.013979] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.013981] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [520.013982] FS: 0000000000000000(0000) GS:ffff8888daf97000(0000) knlGS:0000000000000000 <4> [520.013984] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.013986] CR2: 000073a8a7cc9b00 CR3: 000000000344c003 CR4: 0000000000f72ef0 <4> [520.013988] PKRU: 55555554 <4> [520.013989] Call Trace: <4> [520.013991] <TASK> <4> [520.013995] ? lock_sync+0x100/0x100 <4> [520.014001] ? lock_release+0xd0/0x2b0 <4> [520.014007] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.014013] process_one_work+0x239/0x760 <4> [520.014021] worker_thread+0x200/0x3f0 <4> [520.014024] ? __pfx_worker_thread+0x10/0x10 <4> [520.014026] kthread+0x10d/0x150 <4> [520.014030] ? __pfx_kthread+0x10/0x10 <4> [520.014034] ret_from_fork+0x3d4/0x480 <4> [520.014037] ? __pfx_kthread+0x10/0x10 <4> [520.014041] ret_from_fork_asm+0x1a/0x30 <4> [520.014049] </TASK> <4> [520.014050] irq event stamp: 433701 <4> [520.014052] hardirqs last enabled at (433707): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.014056] hardirqs last disabled at (433712): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.014058] softirqs last enabled at (432578): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.014062] softirqs last disabled at (432571): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.014064] ---[ end trace 0000000000000000 ]--- <6> [520.014066] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.014140] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.014147] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.015420] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.015543] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [520.015651] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.015911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.016005] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.016125] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.016218] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.016315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.016445] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.016535] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.016625] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.016724] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.016832] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.016956] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.017980] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.028712] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.028963] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.030013] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.030099] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.030178] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.030257] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.030334] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.030434] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.030511] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.030587] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.030665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.030753] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.030845] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.030927] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.031003] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.031080] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.031158] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.031233] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.031307] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.031391] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.031488] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.031573] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.031663] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.031747] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.031832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.031910] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.031987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.032065] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.032155] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.032252] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.032343] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.032444] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.032529] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.032610] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.032695] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.032790] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.032889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.032977] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.033056] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.033143] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.033225] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.033305] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.033394] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.033493] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.033577] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.033660] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.033741] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.033825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.033902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.033985] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.034066] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.034165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.034243] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.034331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.034419] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.034506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.034587] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.034668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.034745] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.034834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.034934] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.035172] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.035178] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967177, lrc_seqno=4294967177, guc_id=1, flags=0x4 in no process [-1] <7> [520.035182] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.035246] ------------[ cut here ]------------ <4> [520.035247] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.035249] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#5: kworker/u64:5/204 <4> [520.035321] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.035406] autofs4 [last unloaded: snd_hda_intel] <4> [520.035411] CPU: 5 UID: 0 PID: 204 Comm: kworker/u64:5 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.035415] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.035416] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.035417] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.035423] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.035493] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.035495] RSP: 0018:ffffc90001627ca0 EFLAGS: 00010246 <4> [520.035497] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.035499] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.035500] RBP: ffffc90001627db0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.035501] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.035503] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [520.035504] FS: 0000000000000000(0000) GS:ffff8888daf17000(0000) knlGS:0000000000000000 <4> [520.035506] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.035507] CR2: 000059da42793190 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [520.035509] PKRU: 55555554 <4> [520.035510] Call Trace: <4> [520.035511] <TASK> <4> [520.035515] ? lock_sync+0x100/0x100 <4> [520.035520] ? lock_release+0xd0/0x2b0 <4> [520.035526] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.035531] process_one_work+0x239/0x760 <4> [520.035538] worker_thread+0x200/0x3f0 <4> [520.035541] ? __pfx_worker_thread+0x10/0x10 <4> [520.035543] kthread+0x10d/0x150 <4> [520.035546] ? __pfx_kthread+0x10/0x10 <4> [520.035550] ret_from_fork+0x3d4/0x480 <4> [520.035553] ? __pfx_kthread+0x10/0x10 <4> [520.035558] ret_from_fork_asm+0x1a/0x30 <4> [520.035572] </TASK> <4> [520.035574] irq event stamp: 546905 <4> [520.035576] hardirqs last enabled at (546911): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.035581] hardirqs last disabled at (546916): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.035584] softirqs last enabled at (545386): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.035588] softirqs last disabled at (545377): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.035592] ---[ end trace 0000000000000000 ]--- <7> [520.036866] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.036982] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [520.043270] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.043624] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.043993] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.044142] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.044264] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.044393] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.044514] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.044635] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.044767] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.044888] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.045007] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.045127] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.045279] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.045425] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.045542] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.045658] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.045772] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.045888] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.046019] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.046136] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.046253] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.046374] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.046489] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.046605] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.054611] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.054744] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.054861] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.054989] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.055104] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.055221] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.055336] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.055466] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.055583] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.055723] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.055842] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.055962] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.056101] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.056222] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.056368] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.056483] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.056598] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.056711] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.056825] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.056950] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.057065] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.057179] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.057424] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.057541] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <5> [520.074877] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=3, flags=0x4 in no process [-1] <7> [520.074888] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.075032] ------------[ cut here ]------------ <4> [520.075035] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.075038] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:8/990 <4> [520.075187] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.075321] autofs4 [last unloaded: snd_hda_intel] <4> [520.075330] CPU: 10 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.075336] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.075338] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.075342] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.075352] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.075532] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.075536] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [520.075541] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.075543] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.075546] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.075548] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.075551] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.075553] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [520.075556] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.075559] CR2: 0000799f62a15000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [520.075562] PKRU: 55555554 <4> [520.075564] Call Trace: <4> [520.075567] <TASK> <4> [520.075575] ? lock_sync+0x100/0x100 <4> [520.075585] ? lock_release+0xd0/0x2b0 <4> [520.075596] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.075608] process_one_work+0x239/0x760 <4> [520.075620] worker_thread+0x200/0x3f0 <4> [520.075626] ? __pfx_worker_thread+0x10/0x10 <4> [520.075631] kthread+0x10d/0x150 <4> [520.075637] ? __pfx_kthread+0x10/0x10 <4> [520.075645] ret_from_fork+0x3d4/0x480 <4> [520.075649] ? __pfx_kthread+0x10/0x10 <4> [520.075656] ret_from_fork_asm+0x1a/0x30 <4> [520.075670] </TASK> <4> [520.075673] irq event stamp: 369533 <4> [520.075675] hardirqs last enabled at (369539): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.075681] hardirqs last disabled at (369544): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.075686] softirqs last enabled at (369372): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.075691] softirqs last disabled at (369365): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.075696] ---[ end trace 0000000000000000 ]--- <6> [520.075700] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.075835] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.075850] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.076263] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.076634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.076789] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.076945] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.077093] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.077239] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.077392] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.077540] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.077681] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.077816] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.077962] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.078130] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.079214] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.089971] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 10ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.090246] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.091544] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.091654] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.091761] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.091867] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.091971] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.092076] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.092180] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.092286] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.092411] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.092526] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.092638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.092749] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.092858] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.092968] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.093078] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.093186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.093293] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.093404] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.093506] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.093617] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.093729] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.093835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.093941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.094045] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.094150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.094257] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.094362] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.094466] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.094567] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.094665] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.094761] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.094857] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.094958] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.095057] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.095156] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.095247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.095337] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.095436] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.095523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.095608] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.095698] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.095786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.095871] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.095953] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.096039] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.096121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.096202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.096281] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.096365] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.096441] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.096517] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.096595] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.096673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.096751] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.096832] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.096912] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.096990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.097068] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.097149] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.097408] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.097412] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=3, flags=0x4 in no process [-1] <7> [520.097415] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.097473] ------------[ cut here ]------------ <4> [520.097475] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.097476] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:8/990 <4> [520.097547] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.097622] autofs4 [last unloaded: snd_hda_intel] <4> [520.097626] CPU: 10 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.097629] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.097631] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.097632] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.097638] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.097708] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.097710] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [520.097713] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.097715] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.097716] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.097718] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.097719] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.097720] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [520.097722] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.097724] CR2: 0000799f62a15000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [520.097726] PKRU: 55555554 <4> [520.097727] Call Trace: <4> [520.097728] <TASK> <4> [520.097732] ? lock_sync+0x100/0x100 <4> [520.097737] ? lock_release+0xd0/0x2b0 <4> [520.097743] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.097748] process_one_work+0x239/0x760 <4> [520.097755] worker_thread+0x200/0x3f0 <4> [520.097757] ? __pfx_worker_thread+0x10/0x10 <4> [520.097760] kthread+0x10d/0x150 <4> [520.097763] ? __pfx_kthread+0x10/0x10 <4> [520.097766] ret_from_fork+0x3d4/0x480 <4> [520.097769] ? __pfx_kthread+0x10/0x10 <4> [520.097772] ret_from_fork_asm+0x1a/0x30 <4> [520.097779] </TASK> <4> [520.097781] irq event stamp: 373545 <4> [520.097782] hardirqs last enabled at (373551): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.097785] hardirqs last disabled at (373556): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.097787] softirqs last enabled at (372356): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.097790] softirqs last disabled at (372349): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.097792] ---[ end trace 0000000000000000 ]--- <6> [520.097794] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.097865] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.097871] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.098280] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.098528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.098616] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.098709] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.098799] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.098888] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.098975] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.099062] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.099149] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.099231] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.099327] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.099453] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.100459] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.110361] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.110611] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.111635] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.111715] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.111792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.111869] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.111947] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.112023] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.112100] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.112175] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.112250] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.112329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.112420] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.112499] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.112576] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.112651] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.112727] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.112801] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.112877] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.112952] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.113026] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.113107] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.113191] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.113273] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.113360] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.113439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.113516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.113593] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.113676] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.113760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.113843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.113924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.114006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.114091] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.114181] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.114268] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.114357] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.114439] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.114520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.114601] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.114684] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.114765] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.114851] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.114934] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.115013] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.115090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.115171] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.115249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.115327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.115413] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.115496] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.115575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.115653] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.115734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.115812] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.115889] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.115966] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.116044] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.116121] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.116202] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.116285] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.116530] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.116534] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=3, flags=0x4 in no process [-1] <7> [520.116537] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.116596] ------------[ cut here ]------------ <4> [520.116597] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.116599] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:8/990 <4> [520.116669] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.116735] autofs4 [last unloaded: snd_hda_intel] <4> [520.116739] CPU: 10 UID: 0 PID: 990 Comm: kworker/u64:8 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.116742] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.116743] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.116744] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.116749] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.116819] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.116820] RSP: 0018:ffffc90003657ca0 EFLAGS: 00010246 <4> [520.116823] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.116824] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.116825] RBP: ffffc90003657db0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.116826] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.116828] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.116829] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [520.116831] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.116832] CR2: 0000799f62a15000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [520.116833] PKRU: 55555554 <4> [520.116835] Call Trace: <4> [520.116836] <TASK> <4> [520.116839] ? lock_sync+0x100/0x100 <4> [520.116844] ? lock_release+0xd0/0x2b0 <4> [520.116849] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.116855] process_one_work+0x239/0x760 <4> [520.116861] worker_thread+0x200/0x3f0 <4> [520.116864] ? __pfx_worker_thread+0x10/0x10 <4> [520.116866] kthread+0x10d/0x150 <4> [520.116869] ? __pfx_kthread+0x10/0x10 <4> [520.116873] ret_from_fork+0x3d4/0x480 <4> [520.116874] ? __pfx_kthread+0x10/0x10 <4> [520.116878] ret_from_fork_asm+0x1a/0x30 <4> [520.116885] </TASK> <4> [520.116886] irq event stamp: 377559 <4> [520.116887] hardirqs last enabled at (377565): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.116890] hardirqs last disabled at (377570): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.116892] softirqs last enabled at (376458): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.116895] softirqs last disabled at (376451): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.116897] ---[ end trace 0000000000000000 ]--- <7> [520.117971] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.118233] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [520.124694] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.124995] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.125122] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.125244] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.125370] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.125487] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.125603] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.125718] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.125833] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.125948] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.126064] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.126180] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.126321] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.126451] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.126564] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.126681] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.126797] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.126912] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.127026] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.127138] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.127250] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.127369] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.127483] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.127597] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.168798] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.169025] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.169380] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.169568] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.169743] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.169911] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.170085] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.170249] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.170413] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.170567] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.170725] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.170874] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.171109] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.171255] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.171404] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.171537] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.171672] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.171797] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.171923] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.172053] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.172174] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.172293] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.172416] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.172530] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.177471] xe 0000:03:00.0: [drm:xe_exec_ioctl [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_exec.c:153: q->ops->reset_status(q) <5> [520.184454] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=1, flags=0x4 in no process [-1] <7> [520.184466] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.184664] ------------[ cut here ]------------ <4> [520.184668] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.184672] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#14: kworker/u64:17/5666 <4> [520.184889] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.185062] autofs4 [last unloaded: snd_hda_intel] <4> [520.185073] CPU: 14 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.185080] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.185083] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.185087] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.185101] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.185309] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.185314] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [520.185320] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.185324] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.185327] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.185331] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.185334] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [520.185338] FS: 0000000000000000(0000) GS:ffff8888db397000(0000) knlGS:0000000000000000 <4> [520.185342] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.185346] CR2: 00005c876f15dc78 CR3: 000000000344c006 CR4: 0000000000f72ef0 <4> [520.185350] PKRU: 55555554 <4> [520.185466] Call Trace: <4> [520.185470] <TASK> <4> [520.185483] ? lock_sync+0x100/0x100 <4> [520.185498] ? lock_release+0xd0/0x2b0 <4> [520.185515] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.185532] process_one_work+0x239/0x760 <4> [520.185552] worker_thread+0x200/0x3f0 <4> [520.185560] ? __pfx_worker_thread+0x10/0x10 <4> [520.185567] kthread+0x10d/0x150 <4> [520.185575] ? __pfx_kthread+0x10/0x10 <4> [520.185586] ret_from_fork+0x3d4/0x480 <4> [520.185592] ? __pfx_kthread+0x10/0x10 <4> [520.185602] ret_from_fork_asm+0x1a/0x30 <4> [520.185625] </TASK> <4> [520.185629] irq event stamp: 225819 <4> [520.185632] hardirqs last enabled at (225825): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.185641] hardirqs last disabled at (225830): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.185647] softirqs last enabled at (225732): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.185654] softirqs last disabled at (225621): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.185660] ---[ end trace 0000000000000000 ]--- <6> [520.185666] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.185879] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.185896] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.186291] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.186781] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.187027] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.187277] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.187528] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.187764] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.187987] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.188214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.188448] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.188665] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.188895] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.189138] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.191005] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.201369] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.201814] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.203390] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.203658] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.203920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.204185] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.204636] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.204820] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.204990] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.205151] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.205310] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.205479] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.205626] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.205770] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.205908] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.206045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.206181] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.206319] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.206475] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.206612] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.206749] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.206893] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.207033] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.207169] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.207303] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.207446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.207575] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.207703] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.207825] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.207957] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.208090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.208216] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.208339] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.208476] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.208597] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.208719] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.208840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.208955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.209069] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.209180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.209298] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.209414] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.209523] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.209628] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.209732] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.209833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.209941] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.210047] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.210150] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.210251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.210345] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.210454] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.210549] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.210648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.210744] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.210835] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.210924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.211015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.211103] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.211192] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.211279] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.211636] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.211641] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=1, flags=0x4 in no process [-1] <7> [520.211644] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.211708] ------------[ cut here ]------------ <4> [520.211709] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.211711] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#8: kworker/u64:17/5666 <4> [520.211787] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.211858] autofs4 [last unloaded: snd_hda_intel] <4> [520.211862] CPU: 8 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.211865] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.211867] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.211868] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.211873] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.211943] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.211945] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [520.211947] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.211949] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.211950] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.211951] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.211952] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [520.211954] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [520.211955] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.211957] CR2: 000073a8aaf45c60 CR3: 000000019544f003 CR4: 0000000000f72ef0 <4> [520.211958] PKRU: 55555554 <4> [520.211959] Call Trace: <4> [520.211961] <TASK> <4> [520.211965] ? lock_sync+0x100/0x100 <4> [520.211969] ? lock_release+0xd0/0x2b0 <4> [520.211975] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.211981] process_one_work+0x239/0x760 <4> [520.211988] worker_thread+0x200/0x3f0 <4> [520.211990] ? __pfx_worker_thread+0x10/0x10 <4> [520.211993] kthread+0x10d/0x150 <4> [520.211996] ? __pfx_kthread+0x10/0x10 <4> [520.211999] ret_from_fork+0x3d4/0x480 <4> [520.212002] ? __pfx_kthread+0x10/0x10 <4> [520.212005] ret_from_fork_asm+0x1a/0x30 <4> [520.212012] </TASK> <4> [520.212013] irq event stamp: 229853 <4> [520.212015] hardirqs last enabled at (229859): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.212017] hardirqs last disabled at (229864): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.212020] softirqs last enabled at (228530): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.212022] softirqs last disabled at (228523): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.212025] ---[ end trace 0000000000000000 ]--- <6> [520.212027] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.212098] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.212104] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.212195] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.212456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.212543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.212634] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.212723] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.212813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.212902] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.212990] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.213079] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.213163] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.213259] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.213385] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.214400] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.224361] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.224610] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.225627] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.225707] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.225784] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.225861] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.225939] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.226016] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.226094] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.226171] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.226247] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.226322] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.226406] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.226481] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.226559] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.226638] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.226716] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.226791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.226866] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.226941] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.227015] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.227098] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.227180] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.227261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.227342] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.227435] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.227516] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.227599] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.227678] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.227760] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.227840] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.227920] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.227999] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.228078] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.228163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.228249] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.228333] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.228423] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.228503] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.228584] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.228668] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.228750] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.228834] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.228916] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.228995] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.229077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.229163] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.229245] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.229325] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.229411] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.229491] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.229572] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.229651] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.229733] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.229813] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.229891] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.229970] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.230055] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.230135] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.230214] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.230295] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.230635] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.230640] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967178, lrc_seqno=4294967178, guc_id=1, flags=0x4 in no process [-1] <7> [520.230642] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.230703] ------------[ cut here ]------------ <4> [520.230705] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.230706] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#8: kworker/u64:17/5666 <4> [520.230779] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.230844] autofs4 [last unloaded: snd_hda_intel] <4> [520.230848] CPU: 8 UID: 0 PID: 5666 Comm: kworker/u64:17 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.230851] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.230852] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.230854] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.230859] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.230928] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.230930] RSP: 0018:ffffc9000834fca0 EFLAGS: 00010246 <4> [520.230932] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.230933] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.230935] RBP: ffffc9000834fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.230936] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.230937] R13: ffff888104cd0010 R14: ffff8881853bd018 R15: 00000000ffffffc2 <4> [520.230939] FS: 0000000000000000(0000) GS:ffff8888db097000(0000) knlGS:0000000000000000 <4> [520.230940] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.230941] CR2: 000073a8aaf45c60 CR3: 000000019544f003 CR4: 0000000000f72ef0 <4> [520.230943] PKRU: 55555554 <4> [520.230944] Call Trace: <4> [520.230945] <TASK> <4> [520.230949] ? lock_sync+0x100/0x100 <4> [520.230954] ? lock_release+0xd0/0x2b0 <4> [520.230959] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.230965] process_one_work+0x239/0x760 <4> [520.230971] worker_thread+0x200/0x3f0 <4> [520.230974] ? __pfx_worker_thread+0x10/0x10 <4> [520.230976] kthread+0x10d/0x150 <4> [520.230979] ? __pfx_kthread+0x10/0x10 <4> [520.230983] ret_from_fork+0x3d4/0x480 <4> [520.230985] ? __pfx_kthread+0x10/0x10 <4> [520.230988] ret_from_fork_asm+0x1a/0x30 <4> [520.230996] </TASK> <4> [520.230997] irq event stamp: 233859 <4> [520.230998] hardirqs last enabled at (233865): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.231001] hardirqs last disabled at (233870): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.231003] softirqs last enabled at (232482): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.231006] softirqs last disabled at (232473): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.231008] ---[ end trace 0000000000000000 ]--- <7> [520.231985] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.232103] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [520.238700] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.238903] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.239027] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.239148] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.239266] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.239392] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.239511] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.239629] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.239746] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.240048] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.240164] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.240278] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.240424] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.240540] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.240653] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.240766] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.240879] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.240991] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.241103] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.241216] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.241331] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.241459] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.241573] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.241686] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.284107] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.284465] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.284708] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.284948] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.285182] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.285408] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.285636] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.285850] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.286062] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.286292] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.286502] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.286689] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.286930] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 0, sizes: 8388608 & 8388608 <7> [520.287106] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 1, sizes: 8388608 & 8388608 <7> [520.287280] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 2, sizes: 8388608 & 8388608 <7> [520.287471] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 3, sizes: 8388608 & 8388608 <7> [520.287635] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 4, sizes: 8388608 & 8388608 <7> [520.287790] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 5, sizes: 8388608 & 8388608 <7> [520.287945] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 6, sizes: 8388608 & 8388608 <7> [520.288120] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 7, sizes: 8388608 & 8388608 <7> [520.288268] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 8, sizes: 8388608 & 8388608 <7> [520.288419] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 9, sizes: 8388608 & 8388608 <7> [520.288560] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 10, sizes: 8388608 & 8388608 <7> [520.288724] xe 0000:03:00.0: [drm:__xe_migrate_copy [xe]] Pass 11, sizes: 8388608 & 8388608 <7> [520.292639] xe 0000:03:00.0: [drm:xe_exec_ioctl [xe]] Ioctl argument check failed at drivers/gpu/drm/xe/xe_exec.c:153: q->ops->reset_status(q) <6> [520.299626] [IGT] xe_evict: finished subtest evict-beng-cm-threads-small, FAIL <5> [520.299812] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967179, lrc_seqno=4294967179, guc_id=3, flags=0x4 in no process [-1] <7> [520.299818] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.299902] ------------[ cut here ]------------ <4> [520.299903] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.299905] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:1/118 <4> [520.299990] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.300077] autofs4 [last unloaded: snd_hda_intel] <4> [520.300082] CPU: 1 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.300086] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.300087] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.300089] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.300096] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.300179] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.300181] RSP: 0018:ffffc9000054bca0 EFLAGS: 00010246 <4> [520.300184] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.300186] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.300188] RBP: ffffc9000054bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.300190] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.300191] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.300193] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [520.300195] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.300197] CR2: 000073a8a6e227fd CR3: 000000019544f006 CR4: 0000000000f72ef0 <4> [520.300199] PKRU: 55555554 <4> [520.300201] Call Trace: <4> [520.300202] <TASK> <4> [520.300207] ? lock_sync+0x100/0x100 <4> [520.300213] ? lock_release+0xd0/0x2b0 <4> [520.300220] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.300227] process_one_work+0x239/0x760 <6> [520.300229] [IGT] xe_evict: exiting, ret=98 <4> [520.300234] worker_thread+0x200/0x3f0 <4> [520.300238] ? __pfx_worker_thread+0x10/0x10 <4> [520.300241] kthread+0x10d/0x150 <4> [520.300244] ? __pfx_kthread+0x10/0x10 <4> [520.300248] ret_from_fork+0x3d4/0x480 <4> [520.300251] ? __pfx_kthread+0x10/0x10 <4> [520.300255] ret_from_fork_asm+0x1a/0x30 <4> [520.300263] </TASK> <4> [520.300265] irq event stamp: 702959 <4> [520.300266] hardirqs last enabled at (702965): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.300270] hardirqs last disabled at (702970): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.300272] softirqs last enabled at (701252): [<ffffffff8133ac13>] kernel_fpu_end+0x53/0x70 <4> [520.300275] softirqs last disabled at (701250): [<ffffffff8133b324>] kernel_fpu_begin_mask+0xc4/0x120 <4> [520.300278] ---[ end trace 0000000000000000 ]--- <6> [520.300280] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.300408] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <6> [520.300420] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.300812] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.301077] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <6> [520.301133] Console: switching to colour frame buffer device 240x67 <7> [520.301201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.301315] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.301487] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.301618] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.301730] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.301844] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.301959] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.302044] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.302142] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.302258] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.303269] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.313364] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.313685] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.314917] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.315065] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.315210] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.315349] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.315521] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.315663] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.315791] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.315920] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.316053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.316186] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.316312] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.316456] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.316586] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.316706] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.316822] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.316937] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.317053] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.317170] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.317287] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.317428] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.317561] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.317673] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.317786] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.317897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.318007] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.318116] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.318224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.318331] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.318474] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.318589] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.318700] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.318804] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.318911] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.319018] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.319124] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.319224] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.319321] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.319432] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.319543] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.319643] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.319739] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.319833] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.319925] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.320016] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.320111] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.320200] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.320289] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.320393] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.320478] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.320562] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.320648] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.320734] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.320817] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.320894] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.320972] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.321052] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.321130] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.321206] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.321288] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.321517] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.321522] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967179, lrc_seqno=4294967179, guc_id=3, flags=0x4 in no process [-1] <7> [520.321524] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.321583] ------------[ cut here ]------------ <4> [520.321584] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.321586] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#1: kworker/u64:1/118 <4> [520.321658] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.321726] autofs4 [last unloaded: snd_hda_intel] <4> [520.321730] CPU: 1 UID: 0 PID: 118 Comm: kworker/u64:1 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.321733] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.321734] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.321736] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.321741] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.321811] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.321813] RSP: 0018:ffffc9000054bca0 EFLAGS: 00010246 <4> [520.321815] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.321817] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.321818] RBP: ffffc9000054bdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.321819] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.321821] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.321822] FS: 0000000000000000(0000) GS:ffff8888dad17000(0000) knlGS:0000000000000000 <4> [520.321824] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.321825] CR2: 000073a8a6e227fd CR3: 000000000344c001 CR4: 0000000000f72ef0 <4> [520.321827] PKRU: 55555554 <4> [520.321828] Call Trace: <4> [520.321829] <TASK> <4> [520.321833] ? lock_sync+0x100/0x100 <4> [520.321844] ? lock_release+0xd0/0x2b0 <4> [520.321852] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.321859] process_one_work+0x239/0x760 <4> [520.321868] worker_thread+0x200/0x3f0 <4> [520.321872] ? __pfx_worker_thread+0x10/0x10 <4> [520.321876] kthread+0x10d/0x150 <4> [520.321880] ? __pfx_kthread+0x10/0x10 <4> [520.321885] ret_from_fork+0x3d4/0x480 <4> [520.321888] ? __pfx_kthread+0x10/0x10 <4> [520.321893] ret_from_fork_asm+0x1a/0x30 <4> [520.321904] </TASK> <4> [520.321905] irq event stamp: 707141 <4> [520.321907] hardirqs last enabled at (707147): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.321911] hardirqs last disabled at (707152): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.321915] softirqs last enabled at (705946): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.321918] softirqs last disabled at (705933): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.321922] ---[ end trace 0000000000000000 ]--- <6> [520.321924] xe 0000:03:00.0: [drm] Tile0: GT0: trying reset from guc_exec_queue_timedout_job [xe] <6> [520.322023] xe 0000:03:00.0: [drm] Tile0: GT0: reset queued <5> [520.322036] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967174, lrc_seqno=4294967174, guc_id=6, flags=0x0 in xe_evict [6588] <7> [520.322044] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <5> [520.322397] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=25, flags=0x0 in xe_evict [6588] <7> [520.322401] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <5> [520.322992] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=27, flags=0x0 in xe_evict [6588] <7> [520.322996] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <7> [520.323074] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.323191] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <5> [520.323493] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=29, flags=0x0 in xe_evict [6588] <7> [520.323496] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <5> [520.323588] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=31, flags=0x0 in xe_evict [6588] <7> [520.323592] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <5> [520.323675] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=33, flags=0x0 in xe_evict [6588] <7> [520.323677] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <5> [520.323758] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967173, lrc_seqno=4294967173, guc_id=35, flags=0x0 in xe_evict [6588] <7> [520.323761] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <6> [520.323849] xe 0000:03:00.0: [drm] Tile0: GT0: reset started <7> [520.324092] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel stopped <7> [520.324410] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying GT save-restore MMIOs <7> [520.324520] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x4148] = 0x00000000 <7> [520.324621] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x8828] = 0x00800000 <7> [520.324714] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb0c8] = 0x11111440 <7> [520.324808] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb104] = 0x08104440 <7> [520.324897] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb108] = 0x30200000 <7> [520.324998] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xb158] = 0x0000007f <7> [520.325090] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00000100 <7> [520.325178] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] WOPCM: 4096K <7> [520.325275] xe 0000:03:00.0: [drm:xe_wopcm_init [xe]] GuC WOPCM is already locked [6144K, 832K) <7> [520.325421] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel disabled <7> [520.326754] xe 0000:03:00.0: [drm:xe_guc_ads_populate [xe]] Tile0: GT0: Updated ADS capture size 20480 (was 49152) <7> [520.337361] xe 0000:03:00.0: [drm:guc_wait_ucode [xe]] Tile0: GT0: GuC load: init took 9ms, freq = 2150MHz (req = 2133MHz), before = 2150MHz, status = 0x8002F034 <7> [520.337614] xe 0000:03:00.0: [drm:guc_ct_change_state [xe]] Tile0: GT0: GuC CT communication channel enabled <7> [520.338710] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: flag:0x1 <7> [520.338792] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: mocs entries: 16 <7> [520.338870] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[0] 0x4000 0xc <7> [520.338949] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[1] 0x4004 0x10c <7> [520.339027] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[2] 0x4008 0x130 <7> [520.339103] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[3] 0x400c 0x13c <7> [520.339179] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[4] 0x4010 0x100 <7> [520.339255] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[5] 0x4014 0x100 <7> [520.339329] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[6] 0x4018 0x100 <7> [520.339428] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[7] 0x401c 0x100 <7> [520.339509] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[8] 0x4020 0x100 <7> [520.339588] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[9] 0x4024 0x100 <7> [520.339665] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[10] 0x4028 0x100 <7> [520.339742] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[11] 0x402c 0x100 <7> [520.339818] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[12] 0x4030 0x100 <7> [520.339894] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[13] 0x4034 0x100 <7> [520.339969] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[14] 0x4038 0x100 <7> [520.340045] xe 0000:03:00.0: [drm:xe_mocs_init [xe]] Tile0: GT0: GLOB_MOCS[15] 0x403c 0x100 <7> [520.340119] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying rcs0 save-restore MMIOs <7> [520.340201] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x2050] = 0x10001000 <7> [520.340285] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20c4] = 0x3f7e0306 <7> [520.340372] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x20d4] = 0xc080c080 <7> [520.340456] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d0] = 0x00006210 <7> [520.340534] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d4] = 0x000062a8 <7> [520.340612] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24d8] = 0x1000dafc <7> [520.340691] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24dc] = 0x1000db01 <7> [520.340768] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x24e0] = 0x0000db1c <7> [520.340848] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe194] = 0x00400040 <7> [520.340929] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe48c] = 0x02000200 <7> [520.341008] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe49c] = 0x40004000 <7> [520.341086] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4c4] = 0x10401040 <7> [520.341165] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe4f0] = 0x00020002 <7> [520.341251] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe530] = 0x00000400 <7> [520.341340] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7c8] = 0x04002000 <7> [520.341446] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0xe7cc] = 0x00009100 <7> [520.341530] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x14800] = 0x00020002 <7> [520.341611] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs0 save-restore MMIOs <7> [520.341689] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x22050] = 0x10001000 <7> [520.341767] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220c4] = 0x3f7e0306 <7> [520.341843] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x220d4] = 0xc080c080 <7> [520.341924] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying bcs8 save-restore MMIOs <7> [520.342006] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee050] = 0x10001000 <7> [520.342088] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0c4] = 0x3f7e0306 <7> [520.342166] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x3ee0d4] = 0xc080c080 <7> [520.342247] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs0 save-restore MMIOs <7> [520.342327] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a050] = 0x10001000 <7> [520.342415] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0c4] = 0x3f7e0308 <7> [520.342492] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a0d4] = 0xc080c080 <7> [520.342570] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d0] = 0x1000dafc <7> [520.342646] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d4] = 0x1000db01 <7> [520.342722] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1a4d8] = 0x0000db1c <7> [520.342801] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: Applying ccs1 save-restore MMIOs <7> [520.342878] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c050] = 0x10001000 <7> [520.342955] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0c4] = 0x3f7e0308 <7> [520.343031] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c0d4] = 0xc080c080 <7> [520.343109] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d0] = 0x1000dafc <7> [520.343185] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d4] = 0x1000db01 <7> [520.343261] xe 0000:03:00.0: [drm:xe_reg_sr_apply_mmio [xe]] Tile0: GT0: REG[0x1c4d8] = 0x0000db1c <7> [520.343344] xe 0000:03:00.0: [drm:xe_gt_apply_ccs_mode [xe]] Tile0: GT0: CCS_MODE=fff0fc0 config:00400000, num_engines:1, num_slices:2 <6> [520.343559] xe 0000:03:00.0: [drm] Tile0: GT0: reset done <5> [520.343563] xe 0000:03:00.0: [drm] Tile0: GT0: Timedout job: seqno=4294967179, lrc_seqno=4294967179, guc_id=3, flags=0x4 in no process [-1] <7> [520.343566] xe 0000:03:00.0: [drm:xe_devcoredump [xe]] Multiple hangs are occurring, but only the first snapshot was taken <4> [520.343625] ------------[ cut here ]------------ <4> [520.343626] xe 0000:03:00.0: [drm] Tile0: GT0: VM job timed out on non-killed execqueue <4> [520.343628] WARNING: drivers/gpu/drm/xe/xe_guc_submit.c:1628 at guc_exec_queue_timedout_job+0x352/0x2400 [xe], CPU#10: kworker/u64:4/203 <4> [520.343700] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink <4> [520.343768] autofs4 [last unloaded: snd_hda_intel] <4> [520.343773] CPU: 10 UID: 0 PID: 203 Comm: kworker/u64:4 Tainted: G S U W L 7.0.0-lgci-xe-xe-4916-659eae6a034b720c8-debug+ #1 PREEMPT(lazy) <4> [520.343776] Tainted: [S]=CPU_OUT_OF_SPEC, [U]=USER, [W]=WARN, [L]=SOFTLOCKUP <4> [520.343777] Hardware name: ASUS System Product Name/PRIME Z790-P WIFI, BIOS 1645 03/15/2024 <4> [520.343779] Workqueue: gt-ordered-wq drm_sched_job_timedout [gpu_sched] <4> [520.343784] RIP: 0010:guc_exec_queue_timedout_job+0x35c/0x2400 [xe] <4> [520.343856] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60 <4> [520.343858] RSP: 0018:ffffc9000161fca0 EFLAGS: 00010246 <4> [520.343861] RAX: ffffffffa120464b RBX: 0000000000000000 RCX: 0000000000000000 <4> [520.343862] RDX: ffff888104cd0010 RSI: ffffffffa120464b RDI: ffffffffa1003e90 <4> [520.343864] RBP: ffffc9000161fdb0 R08: 0000000000000000 R09: 0000000000000000 <4> [520.343865] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 <4> [520.343867] R13: ffff888104cd0010 R14: ffff8881853ba018 R15: 00000000ffffffc2 <4> [520.343868] FS: 0000000000000000(0000) GS:ffff8888db197000(0000) knlGS:0000000000000000 <4> [520.343870] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 <4> [520.343872] CR2: 0000799f62a15000 CR3: 000000000344c004 CR4: 0000000000f72ef0 <4> [520.343874] PKRU: 55555554 <4> [520.343875] Call Trace: <4> [520.343877] <TASK> <4> [520.343881] ? lock_sync+0x100/0x100 <4> [520.343886] ? lock_release+0xd0/0x2b0 <4> [520.343892] drm_sched_job_timedout+0x94/0x1a0 [gpu_sched] <4> [520.343898] process_one_work+0x239/0x760 <4> [520.343905] worker_thread+0x200/0x3f0 <4> [520.343908] ? __pfx_worker_thread+0x10/0x10 <4> [520.343911] kthread+0x10d/0x150 <4> [520.343914] ? __pfx_kthread+0x10/0x10 <4> [520.343918] ret_from_fork+0x3d4/0x480 <4> [520.343921] ? __pfx_kthread+0x10/0x10 <4> [520.343924] ret_from_fork_asm+0x1a/0x30 <4> [520.343932] </TASK> <4> [520.343934] irq event stamp: 695855 <4> [520.343935] hardirqs last enabled at (695861): [<ffffffff814aadd9>] __up_console_sem+0x79/0xa0 <4> [520.343938] hardirqs last disabled at (695866): [<ffffffff814aadbe>] __up_console_sem+0x5e/0xa0 <4> [520.343941] softirqs last enabled at (694876): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.343944] softirqs last disabled at (694871): [<ffffffff813d1e9f>] __irq_exit_rcu+0x13f/0x160 <4> [520.343946] ---[ end trace 0000000000000000 ]--- <7> [520.345009] xe 0000:03:00.0: [drm:xe_gt_sriov_pf_config_restart [xe]] PF: Tile0: GT0: pushed 0 skip 24 of 24 VFs configurations <7> [520.345125] xe 0000:03:00.0: [drm:pf_worker_restart_func [xe]] PF: Tile0: GT0: restart completed <7> [520.377522] xe 0000:03:00.0: [drm:drm_client_dev_restore] fbdev: ret=0

<4> [501.251136] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [501.252009] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [517.633917] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [517.634816] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.301982] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.302849] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.329861] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.330032] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.351668] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.351830] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.417310] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.418839] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.444266] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.444438] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.464205] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.464356] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.529273] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.530125] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.555113] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.555261] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.575159] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.575420] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.640351] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.641353] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.664609] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.664844] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.684669] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.684818] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.749189] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.750149] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.776830] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.776985] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.796831] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.796994] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.861093] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.862015] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.888110] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.888261] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.908784] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.908935] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.973250] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [518.974125] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [518.999907] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.000062] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.019878] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.020039] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.084292] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.085797] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.113640] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.113796] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.134271] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.134437] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.199203] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.200060] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.227590] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.227749] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.247714] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.247864] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.311952] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.312997] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.339273] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.339475] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.359913] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.360068] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.424193] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.425042] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.451941] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.452097] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.472752] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.472903] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.536822] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.537729] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.563310] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.563471] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.583194] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.583344] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.648933] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.649988] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.677203] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.677372] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.698339] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.698495] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.762722] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.763630] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.788701] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.788852] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.810026] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.810188] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.874001] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.874888] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.901062] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.901213] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.920949] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.921102] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [519.985161] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [519.986142] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.013803] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.013969] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.035321] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.035493] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.075187] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.075532] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.097547] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.097708] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.116669] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.116819] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.184889] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.185309] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.211787] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.211943] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.230779] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.230928] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.299990] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.300179] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.321658] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.321811] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

<4> [520.343700] Modules linked in: snd_hda_codec_intelhdmi snd_hda_codec_hdmi pmt_crashlog mei_gsc_proxy mei_lb mtd_intel_dg mei_gsc xe drm_gpuvm drm_gpusvm_helper drm_buddy drm_ttm_helper ttm gpu_sched drm_suballoc_helper drm_exec drm_display_helper cec rc_core drm_kunit_helpers i2c_algo_bit kunit intel_rapl_msr intel_rapl_common intel_uncore_frequency intel_uncore_frequency_common intel_tcc_cooling binfmt_misc x86_pkg_temp_thermal intel_powerclamp cmdlinepart hid_generic coretemp spi_nor asus_nb_wmi mei_hdcp mei_pxp asus_wmi mtd sparse_keymap platform_profile wmi_bmof kvm_intel usbhid hid kvm irqbypass snd_intel_dspcfg ghash_clmulni_intel aesni_intel snd_hda_codec snd_hda_core rapl snd_hwdep r8169 intel_cstate video snd_pcm realtek snd_timer nls_iso8859_1 i2c_i801 snd i2c_mux mei_me spi_intel_pci idma64 soundcore i2c_smbus mei spi_intel intel_pmc_core pmt_telemetry pmt_discovery pmt_class intel_pmc_ssram_telemetry wmi intel_vsec pinctrl_alderlake acpi_tad acpi_pad dm_multipath msr nvme_fabrics fuse efi_pstore nfnetlink

<4> [520.343856] Code: 4d 85 ed 75 03 4c 8b 2f 44 0f b6 62 26 0f b6 58 08 e8 98 ed 5e e1 48 89 c6 48 8d 3d ae a4 39 00 0f b6 cb 45 0f b6 c4 4c 89 ea <67> 48 0f b9 3a 80 bd 48 ff ff ff 00 75 26 48 8b 4d 90 48 8b 41 60

Results for igt@xe_evict@evict-beng-cm-threads-small