C# + AOT编译 + Linux环境 —— 完全不靠谱

作者:张富春(ahfuzhang),转载时请注明作者和引用链接,谢谢!


一个简单的 Kestrel 写的后端服务,尝试解决 Coredump 问题和无法捕获的异常,已经两周多了。

今天再次尝试,崩溃在了完全意想不到的位置。

从这个随机状况来看,并非程序 bug , 而是 AOT 编译到 linux 平台这个事情本身就不靠谱。

C# 后端不要使用 AOT 编译!

C# 后端不要使用 AOT 编译!

今天崩溃时的堆栈信息:

text 复制代码
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".

Thread 27 ".NET TP Worker" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7f25167fc6c0 (LWP 61)]
RhpInterfaceDispatch1 () at /__w/1/s/src/runtime/src/coreclr/runtime/amd64/StubDispatch.S:87
warning: 87	/__w/1/s/src/runtime/src/coreclr/runtime/amd64/StubDispatch.S: No such file or directory

===== Program status =====
Last stopped for thread 27 (Thread 0x7f25167fc6c0 (LWP 61)).
	Using the running image of child process 32.
Program stopped at 0x5599eed7e794.
It stopped with signal SIGSEGV, Segmentation fault.

===== Crashed thread =====
[Current thread is 27 (Thread 0x7f25167fc6c0 (LWP 61))]

===== Crash backtrace =====
#0  RhpInterfaceDispatch1 () at /__w/1/s/src/runtime/src/coreclr/runtime/amd64/StubDispatch.S:87
#1  QiWa_framework_QiWa_Mysql_DbConnectionPool_3__GetAsync_d__9<System___Canon__System___Canon__System___Canon>__MoveNext () at /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/ValueTuple.cs:1491
#2  S_P_CoreLib_System_Runtime_CompilerServices_AsyncMethodBuilderCore__Start<Microsoft_AspNetCore_Http_Extensions_Microsoft_AspNetCore_Http_HttpResponseJsonExtensions___WriteAsJsonAsync_g__WriteAsJsonAsyncSlow_5_0_d_1<System___Canon>> (stateMachine=...) at /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:38
#3  QiWa_framework_QiWa_Mysql_DbConnectionPool_3<System___Canon__System___Canon__System___Canon>__GetAsync () at /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/ValueTuple.cs:1491
#4  csharp_backend_CSharpBackend_Global_OnlineUsers__SaveSessionToMysql_d__2__MoveNext (this=...) at /src/server/csharp_backend/src/global/OnlineUserSession.cs:47
#5  S_P_CoreLib_System_Runtime_CompilerServices_AsyncMethodBuilderCore__Start<csharp_backend_CSharpBackend_Global_OnlineUsers__SaveSessionToMysql_d__2> (stateMachine=...) at /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:38
#6  csharp_backend_CSharpBackend_Global_OnlineUsers__SaveSessionToMysql () at /src/server/csharp_backend/src/global/OnlineUserSession.cs:13
#7  csharp_backend_CSharpBackend_Global_OnlineUsers__Store_d__1__MoveNext (this=...) at /src/server/csharp_backend/src/global/OnlineUserSession.cs:35
#8  S_P_CoreLib_System_Runtime_CompilerServices_AsyncMethodBuilderCore__Start<csharp_backend_CSharpBackend_Global_OnlineUsers__Store_d__1> (stateMachine=...) at /_/src/runtime/src/libraries/System.Private.CoreLib/src/System/Runtime/CompilerServices/AsyncMethodBuilderCore.cs:38
#9  csharp_backend_CSharpBackend_Global_OnlineUsers__Store () at /src/server/csharp_backend/src/global/OnlineUserSession.cs:13

AOT 的编译参数如下:

bash 复制代码
PRJ = csharp_backend
RID = linux-x64
TFM = net10.0
BUILD_DIR = /src/build/Release/linux/amd64/
AOT_OBJ_DIR = ./obj/Release/$(TFM)/$(RID)
AOT_NATIVE_OBJ_DIR = $(AOT_OBJ_DIR)/native

# 保留 NativeAOT 调试元数据,方便分析 k8s 中的 core dump。
AOT_CORE_DUMP_FLAGS = \
	-p:DebugSymbols=true \
	-p:DebugType=portable \
	-p:DebuggerSupport=true \
	-p:StackTraceSupport=true \
	-p:IlcGenerateMapFile=true

build_release:
	docker run -it --rm  \
	--platform linux/amd64 \
  --user "$(id -u):$(id -g)" \
	-m 2048m \
	-v "./":/src/ \
	-w /src/server/csharp_backend/ \
	ahfuzhang/csharp-dbg-all-in-one:dotnet10 \
        bash -lc -x ' \
            dotnet restore $(PRJ).csproj -r $(RID) && \
            dotnet clean $(PRJ).csproj -r $(RID) -c Release && \
            dotnet publish $(PRJ).csproj \
            -r $(RID) \
            -p:DefineConstants=UNIX \
            -p:AllowUnsafeBlocks=true \
            -p:PublishAot=true \
            -p:StripSymbols=false \
            -p:InvariantGlobalization=true \
            -p:OptimizationPreference=Speed \
            -p:CppCompilerAndLinker=clang \
            -p:EventSourceSupport=true \
            -p:EmbedAllSources=true \
            $(AOT_CORE_DUMP_FLAGS) \
            --self-contained true \
            -c Release \
            -o $(BUILD_DIR) && \
            cp -f $(AOT_NATIVE_OBJ_DIR)/$(PRJ).map.xml $(BUILD_DIR) && \
            cp -f $(AOT_OBJ_DIR)/$(PRJ).pdb $(BUILD_DIR) && \
            cp csharp_backend.yaml $(BUILD_DIR) '

这种编译器层面的问题,不是开发者靠个人努力可以解决的。

AOT 不靠谱,有两种行为:

1 无规律的程序因为段错误而崩溃;

2 有的代码在编译时丢弃了 try-catch,而有的代码没有;

  • 导致某些异常原本可以在第一现场解决,从而走到补救逻辑 ------ 但是 catch 不住
  • 请参考前一篇:《诡异问题 C# 中用了 MysqlConnector 后,出现无法捕获的异常》 https://www.cnblogs.com/ahfuzhang/p/21920545

因此,幻想破灭了:

  • 想编译一个干净的无很多 so 依赖的二进制文件 (如同 golang 编译的产物一样)
  • 想要你的服务器进程在一个 5mb alpine 镜像中就能运行起来
  • 想要编译期决定,提升程序性能