fix: prevent nvidia-drm unload hangs after GPU failure#1252
Open
hurryman2212 wants to merge 1 commit into
Open
fix: prevent nvidia-drm unload hangs after GPU failure#1252hurryman2212 wants to merge 1 commit into
hurryman2212 wants to merge 1 commit into
Conversation
Detect RM recovery state before nvidia-drm teardown and skip console restoration, display method submission, and GPU progress waits while releasing NVKMS software resources.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem:
After a GPU becomes unresponsive, unloading nvidia_drm still
performs an atomic display shutdown and releases NVKMS modeset
ownership. Releasing modeset ownership attempts to restore the
framebuffer console, submitting display methods to a GPU that can
no longer process them.
The failure was preceded by:
Console restoration then entered nvEvoMakeRoom(), which repeatedly
reported:
Its five-second timeout only prints the error and restarts the
timer; it does not leave the wait loop.
The unload remained stuck in:
The kernel reported:
The NVIDIA modules could no longer be unloaded and reloaded,
leaving a reboot as the only way to restore the driver stack.
Fix:
Query NV2080_CTRL_CMD_GPU_GET_RECOVERY_ACTION before nvidia_drm
teardown. When RM reports that recovery is required, stop new
event handling, cancel pending hotplug work, and skip atomic
display shutdown and framebuffer console restoration.
Mark NVKMS for recovery teardown before releasing modeset
ownership, then avoid display method submission and GPU progress
waits while continuing to release software resources and RM
handles.
Keep the existing teardown path when RM reports no recovery action
or does not support the recovery-action query.