- Frozen appearence (short or long period)
One of the most common stability problems in graphics occurs when a computer “hangs” or appears completely “frozen” while, in reality, it is processing an end-user command or operation. The end-user typically waits a few seconds and then decides to reboot the computer. The frozen appearance of the computer typically occurs because the GPU is busy processing intensive graphical operations, typically during gameplay. The GPU does not update the display screen, and the computer appears frozen.
In Windows Vista and later, the operating system attempts to detect situations in which computers appear to be completely “frozen”. The operating system then attempts to dynamically recover from the frozen situations so that desktops are responsive again. This process of detection and recovery is known as timeout detection and recovery (TDR). In the TDR process, the operating system’s GPU scheduler calls the display miniportdriver’s DxgkDdiResetFromTimeout function to reinitialize the driver and reset the GPU. Therefore, end users are not required to reboot the operating system, which greatly enhances their experience.
The only visible artifact from the hang detection to the recovery is a screen flicker. This screen flicker results when the operating system resets some portions of the graphics stack, which causes a screen redraw. This flicker is eliminated if the display miniport driver complies with Windows Display Driver Model (WDDM) 1.2 and later. Some legacy Microsoft DirectX applications (for example, those DirectX applications that conform to DirectX versions earlier than 9.0) might render to a black screen at the end of this recovery. The end user would have to restart these applications.
This sequence briefly describes the TDR process:
Timeout detection in the Windows Display Driver Model (WDDM)
The GPU scheduler, which is part of the DirectX graphics kernel subsystem (Dxgkrnl.sys), detects that the GPU is taking more than the permitted amount of time to execute a particular task. The GPU scheduler then tries to preempt this particular task. The preempt operation has a “wait” timeout, which is the actual TDR timeout. This step is thus the timeout detection phase of the process. The default timeout period in Windows operating systems is 2 seconds. If the GPU cannot complete or preempt the current task within the TDR timeout period, the operating system diagnoses that the GPU is frozen.
To prevent timeout detection from occurring, hardware vendors should ensure that graphics operations (that is, direct memory access (DMA) buffer completion) take no more than 2 seconds in end-user scenarios such as productivity and game play.
Preparation for recovery
The operating system’s GPU scheduler calls the display miniport driver’s DxgkDdiResetFromTimeout function to inform the driver that the operating system detected a timeout. The driver must then reinitialize itself and reset the GPU. In addition, the driver must stop accessing memory and should not access hardware. The operating system and the driver collect hardware and other state information that could be useful for post-mortem diagnosis.
The operating system resets the appropriate state of the graphics stack. The video memory manager, which is also part of Dxgkrnl.sys, purges all allocations from video memory. The display miniport driver resets the GPU hardware state. The graphics stack takes the final actions and restores the desktop to the responsive state. As previously mentioned, some legacy DirectX applications might render just black at the end of this recovery, which requires the end user to restart these applications. Well-written DirectX 9Ex and DirectX 10 and later applications that handle Device Remove technology continue to work correctly. An application must release and then re-create its Microsoft Direct3D device and all of the device’s objects. For more information about how DirectX applications recover, see the Windows SDK.
Limiting Repetitive GPU Hangs and Recoveries
Beginning with Windows Vista with Service Pack 1 (SP1) and Windows Server 2008, the user experience has been improved in situations where the GPU hangs frequently and rapidly. Repetitive GPU hangs indicate that the graphics hardware has not recovered successfully. In these situations, the end user must shut down and restart the operating system to fully reset the graphics hardware. If the operating system detects that six or more GPU hangs and subsequent recoveries occur within 1 minute, the operating system bug-checks the computer on the next GPU hang.
TDR Error Messaging
The operating system also logs the preceding message in the Event Viewer application and collects diagnosis information in the form of a debug report. If the end user opted in to provide feedback, the operating system returns this debug report to Microsoft through the Online Crash Analysis (OCA) mechanism.
Timeout Detection and Recovery (TDR) Registry Keys
You can use the following TDR (timeout detection and recovery)-related registry keys for testing or debugging purposes only. That is, they should not be manipulated by any applications outside targeted testing or debugging.
- TdrLevelSpecifies the initial level of recovery. The default value is to recover on timeout (TdrLevelRecover).
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrLevel ValueType : REG_DWORD ValueData : TdrLevelOff (0) - Detection disabled TdrLevelBugcheck (1) - Bug check on detected timeout, for example, no recovery. TdrLevelRecoverVGA (2) - Recover to VGA (not implemented). TdrLevelRecover (3) - Recover on timeout. This is the default value.
- TdrDelaySpecifies the number of seconds that the GPU can delay the preempt request from the GPU scheduler. This is effectively the timeout threshold. The default value is 2 seconds.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrDelay ValueType : REG_DWORD ValueData : Number of seconds to delay. 2 seconds is the default value.
- TdrDdiDelaySpecifies the number of seconds that the operating system allows threads to leave the driver. After a specified time, the operating system bug-checks the computer with the code VIDEO_TDR_FAILURE (0x116). The default value is 5 seconds.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrDdiDelay ValueType : REG_DWORD ValueData : Number of seconds to leave the driver. 5 seconds is the default value.
- TdrTestModeReserved. Do not use.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrTestMode ValueType : REG_DWORD ValueData : Do not use.
- TdrDebugModeSpecifies the debugging-related behavior of the TDR process. The default value is TDR_DEBUG_MODE_RECOVER_NO_PROMPT, which indicates not to break into the debugger.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrDebugMode ValueType : REG_DWORD ValueData : TDR_DEBUG_MODE_OFF (0) - Break to kernel debugger before the recovery to allow investigation of the timeout. TDR_DEBUG_MODE_IGNORE_TIMEOUT (1) - Ignore any timeout. TDR_DEBUG_MODE_RECOVER_NO_PROMPT (2) - Recover without breaking into the debugger. This is the default value. TDR_DEBUG_MODE_RECOVER_UNCONDITIONAL (3) - Recover even if some recovery conditions are not met (for example, recover on consecutive timeouts).
- TdrLimitTimeSupported in Windows Server 2008 and later versions, and Windows Vista with Service Pack 1 (SP1) and later versions. Specifies the default time within which a specific number of TDRs (specified by the TdrLimitCount key) are allowed without crashing the computer. The default value is 60 seconds.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrLimitTime ValueType : REG_DWORD ValueData : Number of seconds before crashing. 60 seconds is the default value.
- TdrLimitCountSupported in Windows Server 2008 and later versions, and Windows Vista with Service Pack 1 (SP1) and later versions. Specifies the default number of TDRs (0x117) that are allowed during the time specified by the TdrLimitTime key without crashing the computer. The default value is 5.
KeyPath : registry HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\GraphicsDrivers KeyValue : TdrLimitCount ValueType : REG_DWORD ValueData : Number of TDRs before crashing. The default value is 5.
(1) Run Windows Update
(2) Update your GPU drivers
(3) If the event occurs (after updating Windows and GPU drivers)more than frequently, you may want to add a TdrLevel key:
- exit all Windows based programs,
- click on the Windows Start button,
- type regedit in the Search box,
- double-click regedit.exe from the results above,
- browse to and then click the following registry subkey: HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\GraphicsDrivers
- on the Edit menu,
- right click New,
- select the QWORD (64-bit) value from the drop-down menu,
- type ‘TdrLevel’ as the Name and click Enter,
- double-click TdrLevel and set the value as 0 (=default)
- click OK,
- close the registry editor and restart your computer for the changes to take effect!
- If this added registry key does not solve the issue, remove TdrLevel.