一:背景
1. 讲故事
今年以来不知道为啥总有些朋友加我微信,让我帮忙分析下操作系统蓝屏问题,我也觉得挺好奇的,就问了其中一位朋友,说是B站来的,我就在拼命回忆,为啥会找我分析蓝屏?突然想到了去年好像录了一集关于 CrowdStrike
的热点事件。。。我本来是做.NET程序
的故障分析,这操作系统的蓝屏虽然不是我擅长的,但也能硬着头皮上吧。
这位朋友找到我的时候,说它已经重装过操作系统,但还是会偶发性蓝屏,让我看下到底是啥原因导致的,那就上 windbg 分析吧。
二:为什么会蓝屏
1. 到底崩溃在哪里
要想找到崩溃点,其实 windbg 提供了一个牛叉的自动化分析命令!analyze -v
,它可以帮我们挖出崩溃上下文,输出如下:
C#
14: kd> !analyze -v
*******************************************************************************
* *
* Bugcheck Analysis *
* *
*******************************************************************************
Unknown bugcheck code (c0000102)
Unknown bugcheck description
Arguments:
Arg1: ffffab027a2c6030
Arg2: ffffab02785d06a0
Arg3: 0000000000000000
Arg4: 0000000000000000
PROCESS_NAME: csrss.exe
STACK_TEXT:
ffff930a`26342578 fffff800`ef921e61 : 00000000`0000004c 00000000`c0000102 ffff930a`2733f390 ffff9703`d52fc310 : nt!KeBugCheckEx
ffff930a`26342580 fffff800`ef93266d : ffffffff`ffffffff ffff930a`26342699 ffffffff`80000a04 00000000`00000000 : nt!PopGracefulShutdown+0x2d9
ffff930a`263425e0 fffff800`ef9101e0 : 00000000`00000001 fffff800`00000000 00000000`00000000 00000000`00000000 : nt!PopTransitionSystemPowerStateEx+0x895
ffff930a`26342700 fffff800`ef8632bd : 00000000`00000047 00000000`ffffff00 00000000`00000001 00000000`00000000 : nt!PopTransitionSystemPowerState+0x4c
ffff930a`263428b0 fffff800`ef862e80 : 00000000`00000000 ffff9703`00000005 00000000`00000001 00000000`c0000004 : nt!PopIssueActionRequest+0x2a9
ffff930a`26342950 fffff800`ef255d3e : 00000000`00000001 00000000`00000000 00000000`fffffffb ffff9703`c5620aa0 : nt!PopPolicyWorkerAction+0x80
ffff930a`263429d0 fffff800`ef1704a2 : ffff9703`00000000 ffff9703`c57d4040 ffff930a`26342b00 ffff9703`c5620aa0 : nt!PopPolicyWorkerThread+0x7e
ffff930a`26342a00 fffff800`ef25585a : ffff9703`c57d4040 ffff9703`c57d4040 fffff800`ef1702f0 ffff9703`c5620aa0 : nt!ExpWorkerThread+0x1b2
ffff930a`26342bb0 fffff800`ef47ae54 : ffffd901`dbd90180 ffff9703`c57d4040 fffff800`ef255800 00000000`00000000 : nt!PspSystemThreadStartup+0x5a
ffff930a`26342c00 00000000`00000000 : ffff930a`26343000 ffff930a`2633c000 00000000`00000000 00000000`00000000 : nt!KiStartSystemThread+0x34
SYMBOL_NAME: nt!PopTransitionSystemPowerStateEx+895
MODULE_NAME: nt
IMAGE_NAME: ntkrnlmp.exe
IMAGE_VERSION: 10.0.26100.2605
...
从卦中可以看到两点信息:
- bugcode=c0000102
微软定义的蓝屏数字都比较小,所以看到这个码也挺奇怪的,从微软的msdn一查原来是NTSTATUS值,它表示 STATUS_FILE_CORRUPT_ERROR
即文件损坏。参见:https://learn.microsoft.com/zh-cn/troubleshoot/azure/virtual-machines/windows/error-code-0xc0000102-status-file-corrupt
- PopGracefulShutdown
从函数的前缀 Pop 可以看出,它是属于内核 电源管理执行体
,翻译过来应该叫 Power on Plug
,正在做系统的关机操作,当然这里的关机有可能是朋友做了 Close
或者 Restart
操作。
为了能够知己知彼,得需要观察下 PopGracefulShutdown 方法都在干嘛?
2. PopGracefulShutdown 在干嘛
要想找到这个答案,可以在 Reactos 中寻找答案,参考代码如下:
C#
VOID
NTAPI
PopGracefulShutdown(IN PVOID Context)
{
PEPROCESS Process = NULL;
/* Process the registered waits and work items */
PopProcessShutDownLists();
/* Loop every process */
Process = PsGetNextProcess(Process);
while (Process)
{
/* Make sure this isn't the idle or initial process */
if ((Process != PsInitialSystemProcess) && (Process != PsIdleProcess))
{
/* Print it */
DPRINT1("%15s is still RUNNING (%p)\n", Process->ImageFileName, Process->UniqueProcessId);
}
/* Get the next process */
Process = PsGetNextProcess(Process);
}
/* First, the HAL handles any "end of boot" special functionality */
DPRINT("HAL shutting down\n");
HalEndOfBoot();
/* Shut down the Shim cache if enabled */
ApphelpCacheShutdown();
/* In this step, the I/O manager does first-chance shutdown notification */
DPRINT("I/O manager shutting down in phase 0\n");
IoShutdownSystem(0);
/* In this step, all workers are killed and hives are flushed */
DPRINT("Configuration Manager shutting down\n");
CmShutdownSystem();
/* Shut down the Executive */
DPRINT("Executive shutting down\n");
ExShutdownSystem();
/* Note that modified pages should be written here (MiShutdownSystem) */
MmShutdownSystem(0);
/* Flush all user files before we start shutting down IO */
/* This is where modified pages are written back by the IO manager */
CcShutdownSystem();
/* In this step, the I/O manager does last-chance shutdown notification */
DPRINT("I/O manager shutting down in phase 1\n");
IoShutdownSystem(1);
CcWaitForCurrentLazyWriterActivity();
/* FIXME: Calling Mm shutdown phase 1 here to get page file dereference
* but it shouldn't be called here. Only phase 2 should be called.
*/
MmShutdownSystem(1);
/* Note that here, we should broadcast the power IRP to devices */
/* In this step, the HAL disables any wake timers */
DPRINT("Disabling wake timers\n");
HalSetWakeEnable(FALSE);
/* And finally the power request is sent */
DPRINT("Taking the system down\n");
PopShutdownSystem(PopAction.Action);
}
从卦中的源代码看,这个关机操作主要是用来关闭各种执行体,那到底是哪一句导致的 KeBugCheckEx
呢?这就需要从汇编上寻找答案,使用 ub 命令即可。
C#
14: kd> ub fffff800`ef921e61
nt!PopGracefulShutdown+0x2b5:
fffff800`ef921e3d 488b0dbc843e00 mov rcx,qword ptr [nt!PopAction+0x40 (fffff800`efd0a300)]
fffff800`ef921e44 488b4140 mov rax,qword ptr [rcx+40h]
fffff800`ef921e48 4c8b4938 mov r9,qword ptr [rcx+38h]
fffff800`ef921e4c 4c8b4130 mov r8,qword ptr [rcx+30h]
fffff800`ef921e50 488b5128 mov rdx,qword ptr [rcx+28h]
fffff800`ef921e54 8b4920 mov ecx,dword ptr [rcx+20h]
fffff800`ef921e57 4889442420 mov qword ptr [rsp+20h],rax
fffff800`ef921e5c e86f6799ff call nt!KeBugCheckEx (fffff800`ef2b85d0)
从汇编中可以看到 KeBugCheckEx 中的各种值是来自于 PopAction
结构体,这个结构体是关机操作的一个上下文类 _POP_POWER_ACTION
,接下来根据汇编代码用 dt 给导出来,参考如下:
C#
14: kd> dt nt!_POP_POWER_ACTION fffff800`efd0a300-0x40
+0x000 Updates : 0 ''
+0x001 State : 0x3 ''
+0x002 Shutdown : 0x1 ''
+0x004 Action : 5 ( PowerActionShutdownReset )
+0x008 LightestState : 6 ( PowerSystemShutdown )
+0x00c Flags : 0xc0000004
+0x010 Status : 0n0
+0x014 DeviceType : 4 ( PolicyInitiatePowerActionAPI )
+0x018 DeviceTypeFlags : 0
+0x020 RequestorInformation : 0xffffab02`785e2b50 _DIAGNOSTIC_BUFFER
+0x028 IrpMinor : 0x2 ''
+0x029 Waking : 0 ''
+0x02c SystemState : 6 ( PowerSystemShutdown )
+0x030 NextSystemState : 1 ( PowerSystemWorking )
+0x034 EffectiveSystemState : 6 ( PowerSystemShutdown )
+0x038 CurrentSystemState : 1 ( PowerSystemWorking )
+0x040 ShutdownBugCode : 0xffff930a`2733f250 _POP_SHUTDOWN_BUG_CHECK
+0x048 DevState : 0xffff9703`d2bf71d0 _POP_DEVICE_SYS_STATE
+0x050 HiberContext : (null)
+0x058 WakeTime : 0
+0x060 SleepTime : 0
+0x068 WakeFirstUnattendedTime : 0
+0x070 WakeAlarmSignaled : 0 ( PoAc )
+0x078 WakeAlarm : [3] <unnamed-tag>
+0x0c0 WakeAlarmPaused : 0 ''
+0x0c8 WakeAlarmLastTime : 0
+0x0d0 DozeDeferralStartTime : 0
+0x0d8 FilteredCapabilities : SYSTEM_POWER_CAPABILITIES
+0x128 WatchdogLock : 0
+0x130 WatchdogDpc : _KDPC
+0x170 WatchdogTimer : _KTIMER
+0x1b0 WatchdogInitialized : 0x1 ''
+0x1b4 WatchdogState : 0 ( PopPowerActionWatchdogStateDisabled )
+0x1b8 WatchdogStartTime : 0
+0x1c0 WatchdogTimeout : 0
+0x1c8 ActionWorkerThread : 0xffff9703`c57d4040 _KTHREAD
+0x1d0 PromoteActionWorkerThread : (null)
+0x1d8 UnlockAfterSleepWorkerThread : (null)
14: kd> dx -id 0,0,ffff9703d27541c0 -r1 ((ntkrnlmp!_POP_SHUTDOWN_BUG_CHECK *)0xffff930a2733f250)
((ntkrnlmp!_POP_SHUTDOWN_BUG_CHECK *)0xffff930a2733f250) : 0xffff930a2733f250 [Type: _POP_SHUTDOWN_BUG_CHECK *]
[+0x000] InitiatingThread : 0xffff9703d274f140 [Type: _ETHREAD *]
[+0x008] InitiatingProcess : 0xffff9703d27541c0 [Type: _EPROCESS *]
[+0x010] ThreadId : 0x4dc [Type: void *]
[+0x018] ProcessId : 0x4d8 [Type: void *]
[+0x020] Code : 0x4c [Type: unsigned long]
[+0x028] Parameter1 : 0xc0000102 [Type: unsigned __int64]
[+0x030] Parameter2 : 0xffff930a2733f390 [Type: unsigned __int64]
[+0x038] Parameter3 : 0xffff9703d52fc310 [Type: unsigned __int64]
[+0x040] Parameter4 : 0xffff9703d53bdcc0 [Type: unsigned __int64]
从卦中可以看到如下信息:
- Action=5
原来这是一个 PowerActionShutdownReset
操作,即 重启 操作。
- Code=0x4c
从 _POP_SHUTDOWN_BUG_CHECK 结构体中可以看到,真正的 BugCheck=0x4c,而导致 0x4c 的真正原因是 0xc0000102,这个在 !anlayze -v
中没有体现出来。
顺带提醒一下,也可以用 kb 2
的 Args to Child
区域直接观察父子异常码。
C#
14: kd> kb 2
# RetAddr : Args to Child : Call Site
00 fffff800`ef921e61 : 00000000`0000004c 00000000`c0000102 ffff930a`2733f390 ffff9703`d52fc310 : nt!KeBugCheckEx
01 fffff800`ef93266d : ffffffff`ffffffff ffff930a`26342699 ffffffff`80000a04 00000000`00000000 : nt!PopGracefulShutdown+0x2d9
3. 到底是什么文件损坏了
在 reactos 中没有找到对 _POP_SHUTDOWN_BUG_CHECK 结构的相关赋值逻辑,所以此时只能在几个 Parameter
参数上下手,可以 du 观察这几个参数。
C#
14: kd> db 0xffff930a2733f390
ffff930a`2733f390 30 60 2c 7a 02 ab ff ff-a0 06 5d 78 02 ab ff ff 0`,z......]x....
ffff930a`2733f3a0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
ffff930a`2733f3b0 00 00 00 00 00 00 00 00-20 25 73 20 25 73 0a 00 ........ %s %s..
ffff930a`2733f3c0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
ffff930a`2733f3d0 00 00 00 00 00 00 00 00-00 00 00 00 00 00 00 00 ................
ffff930a`2733f3e0 54 68 65 20 66 69 6c 65-20 6f 72 20 64 69 72 65 The file or dire
ffff930a`2733f3f0 63 74 6f 72 79 20 73 78-73 73 72 76 20 69 73 20 ctory sxssrv is
ffff930a`2733f400 63 6f 72 72 75 70 74 20-61 6e 64 20 75 6e 72 65 corrupt and unre
14: kd> db 0xffff9703d52fc310
ffff9703`d52fc310 0a 53 54 4f 50 3a 20 63-30 30 30 30 31 30 32 20 .STOP: c0000102
ffff9703`d52fc320 7b 43 6f 72 72 75 70 74-20 46 69 6c 65 7d 0a 00 {Corrupt File}..
ffff9703`d52fc330 a6 01 03 02 44 50 44 69-71 d0 64 bc 00 00 00 00 ....DPDiq.d.....
ffff9703`d52fc340 5c 00 44 00 72 00 69 00-76 00 65 00 72 00 5c 00 \.D.r.i.v.e.r.\.
ffff9703`d52fc350 41 00 6d 00 64 00 50 00-50 00 4d 00 00 00 00 00 A.m.d.P.P.M.....
ffff9703`d52fc360 b7 01 03 02 44 50 44 69-3d d5 a1 9c 00 00 00 00 ....DPDi=.......
ffff9703`d52fc370 5c 00 44 00 72 00 69 00-76 00 65 00 72 00 5c 00 \.D.r.i.v.e.r.\.
ffff9703`d52fc380 61 00 6d 00 64 00 69 00-32 00 63 00 00 00 00 00 a.m.d.i.2.c.....
14: kd> da /c100 0xffff9703d53bdcc0
ffff9703`d53bdcc0 "The file or directory sxssrv is corrupt and unreadable...Please run the Chkdsk utility..."
从 Parameter4
上可以看到原来是 sxssrv.dll
损坏了,通过网络查了下是个 windows子系统文件,所以这个是罪魁祸首,截图如下:
到这里我们大概就捋出来了,原来是客户在电脑重启的过程中,操作系统发现 sxssrv.dll
文件有损,而它又是操作系统非常重要的一个系统文件,所以就蓝屏了。
4. 如何解决
- 使用
sfc /scannow
修复受保护的系统文件,并自动修复。 - 使用
DISM /Online /Cleanup-Image /RestoreHealth
修复系统镜像。
如果上面都搞不定,并且重装系统
也不行,那大概率就是内存或者硬盘的问题了。
- 使用 diskgenius 工具的 坏道检测和修复功能,观察硬盘的健康状态。
- 使用内置的
Windows内存诊断
工具诊断内存健康度。
如果有问题,那该换的就换。
三:总结
操作系统的代码固若金汤,我们常人基本上遇不到代码导致的bug,更多的是第三方配件导致的不给力,解决此类问题大多都是修复,更换。。。