Android 16 显示系统 | 从View 到屏幕系列 - 7 | SurfaceFling Commit

写在前面

SurfaceFlinger 的核心职责：Commit 和 Composite

Commit：

处理上层 App 发起的事务请求，包括:

• 图层的位置、大小、透明度、Z-Order 等状态
• 设置新的 Buffer (如 setBuffer 请求)

SurfaceFlinger 会处理这些事务请求，然后对对应的图层进行更新。为后续的 Compostion 做准备

Composite

对所有的图层进行合成,类似于：

考虑到 SurfaceFlinger 的代码量非常大，我不可能去看 SurfaceFlinger 的每一行代码，这里只抓住主要矛盾：App 渲染好的 Buffer 是更新到 SurfaceFlinger 的 Layer 中

APP 发起事务请求

在前面小节提到了，App 渲染完成之后，会把 Buffer 插入 BLASTBufferQueue 的队列中，然后由 BLASTBufferQueue 发起一个 Transaction 请求

SurfaceComposerClient::Transaction localTransaction;
SurfaceComposerClient::Transaction* t = &localTransaction;
    t->setBuffer(mSurfaceControl, buffer, fence, bufferItem.mFrameNumber, mProducerId,
                 releaseBufferCallback, dequeueTime);
    ...
    t->setApplyToken(mApplyToken).apply(false, true);

先分析 setBuffer 的实现

SurfaceComposerClient::Transaction& SurfaceComposerClient::Transaction::setBuffer(
        const sp<SurfaceControl>& sc, const sp<GraphicBuffer>& buffer,
        const std::optional<sp<Fence>>& fence, const std::optional<uint64_t>& optFrameNumber,
        uint32_t producerId, ReleaseBufferCallback callback, nsecs_t dequeueTime) {
    // 1. 获取 layer_state_t 对象, 传入的参数 sc 是 SurfaceControl
    layer_state_t* s = getLayerState(sc);

    releaseBufferIfOverwriting(*s);

    std::shared_ptr<BufferData> bufferData = std::make_shared<BufferData>();
    bufferData->buffer = buffer;
    ...
    // 设置 layer_state_t 的 what 字段
    s->what |= layer_state_t::eBufferChanged;
    // 设置 layer_state_t 的 bufferData
    s->bufferData = std::move(bufferData);
    ...
}

这里引入了一个重要的数据结构 layer_state_t，Android 源码的注释非常清晰

/*
 * Used to communicate layer information between SurfaceFlinger and its clients.
 */
 struct layer_state_t {
    ...
    sp<IBinder> surface; // Layer 的句柄，用于标识要操作的 Layer
    uint64_t what; // 标识本次 Transaction 中哪些属性被设置，如 位置，大小，透明度等
    // x,y 标识图层的坐标
    float x;
    float y;
    // z 代表 Z-order 层级，决定了显示的顺序
    int32_t z;
    ...
 }

layer_state_t 被用于在 App 和 SurfaceFlinger 传递图层的所有信息, 源码中我只是列举了几个，完整的大家可以自己去看源码。

那接下来我们看 getLayerState 是如何实现的：

layer_state_t* TransactionState::getLayerState(const sp<SurfaceControl>& sc) {
    // 获取 SurfaceControl 的 LayerHandle
    auto handle = sc->getLayerStateHandle();
    if (auto it = std::find_if(mComposerStates.begin(), mComposerStates.end(),
                               [&handle](const auto& composerState) {
                                   return composerState.state.surface == handle;
                               });
        it != mComposerStates.end()) {
        return &it->state;
    }

    // we don't have it, add an initialized layer_state to our list
    ComposerState s;
    // 给 layer_state_t.surface 赋值
    s.state.surface = handle;
    s.state.layerId = sc->getLayerId();
    mComposerStates.push_back(s);
    return &mComposerStates.back().state;
}

1. 通过 getLayerStateHandle 获取 SurfaceControl 的成员变量 handle ，为什么 SurfaceControl 会持有 handle，可以回顾前面提到的 SurfaceControl 和 Layer 的创建过程：juejin.cn/post/752699...
2. 在 juejin.cn/post/752699... 中提到过，通过 handle 可以找到对应的 Layer，所以后续可以通过 layer_state_t.surface 找到需要对应的 Layer。

总结一下，setbuffer 完成了 Transaction 提交的前期准备，填充了 layer_state_t 这个数据结构。

接下来分析 apply 的实现。

SurfaceComposerClient::Transaction::apply 会通过 Binder 跨进程调用 SurfaceFlinger 的 setTransactionState

status_t SurfaceComposerClient::Transaction::apply(bool synchronous, bool oneWay) {
    ...
    // 拿到 SurfaceFlinger 的 binder 句柄
    sp<ISurfaceComposer> sf(ComposerService::getComposerService());
    // 通过 Binder 跨进程调用 SurfaceFlinger 的 setTransactionState 方法
    status_t binderStatus = sf->setTransactionState(std::move(mState));
    ...
}

到这里，就从 App 进程就转到了 SurfaceFlinger 运行的进程了。

SurfaceFlinger Pre-Commit

status_t SurfaceFlinger::setTransactionState(TransactionState&& transactionState) {
    SFTRACE_CALL();
    // 对调用 App 进行权限检查，基于传统 Linux 的 UGO 权限模型
    IPCThreadState* ipc = IPCThreadState::self();
    const int originPid = ipc->getCallingPid();
    const int originUid = ipc->getCallingUid();
    uint32_t permissions = LayerStatePermissions::getTransactionPermissions(originPid, originUid);
    ...
    // 1. 创建一个 ResolvedComposerState 类型的容器
    std::vector<ResolvedComposerState> resolvedStates;
    resolvedStates.reserve(transactionState.mComposerStates.size());
    // 遍历 App 端传过来的 transactionState
    for (auto& state : transactionState.mComposerStates) {
        // 把 App 端传过来的 transactionState所有元素都转移到 resolvedStates
        resolvedStates.emplace_back(std::move(state));
        auto& resolvedState = resolvedStates.back();
        resolvedState.layerId = LayerHandle::getLayerId(resolvedState.state.surface);
        if (resolvedState.state.hasBufferChanges() && resolvedState.state.hasValidBuffer() &&
            resolvedState.state.surface) {
            sp<Layer> layer = LayerHandle::getLayer(resolvedState.state.surface);
            std::string layerName =
                    (layer) ? layer->getDebugName() : std::to_string(resolvedState.state.layerId);
            // externalTexture 是 GraphicBuffer 的一个 wrapper
            resolvedState.externalTexture =
                    getExternalTextureFromBufferData(*resolvedState.state.bufferData,
                                                     layerName.c_str(), transactionState.getId());
            if (resolvedState.externalTexture) {
                resolvedState.state.bufferData->buffer = resolvedState.externalTexture->getBuffer();

            }
        }
    }
    // 2.构建一个 TransactionState, 并插入 Transaction 队列
    QueuedTransactionState state{std::move(transactionState),
                                 std::move(resolvedStates),
                                 std::move(uncacheBufferIds),
                                 postTime,
                                 originPid,
                                 originUid};
    state.workloadHint = queuedWorkload;

    if (mTransactionTracing) {
        mTransactionTracing->addQueuedTransaction(state);
    }
    ...
    return NO_ERROR;
}

在 SurfaceFlinger::setTransactionState 中：

1. 创建了一个 ResolvedComposerState 类型的容器，并且把 App 端传过来的数据都转移到这个容器中

class ResolvedComposerState : public ComposerState {
public:
    ...
    std::shared_ptr<renderengine::ExternalTexture> externalTexture;
    ...
};

ResolvedComposerState 中持有一个 externalTexture, 是 GraphicBuffer 的一个 wrapper 同时 ResolvedComposerState 继承于 ComposerState

class ComposerState {
public:
    layer_state_t state;
    ...
};

在 ComposerState 也持有 layer_state_t 这个数据结构，上面已经提到，layer_state_t 被用于在 App 和 SurfaceFlinger 传递图层的所有信息。

2. 构建一个 QueuedTransactionState 并插入 mTransactionHandler 的 Transaction 队列中

void TransactionHandler::queueTransaction(QueuedTransactionState&& state) {
    mLocklessTransactionQueue.push(std::move(state));
    mPendingTransactionCount.fetch_add(1);
}

这与 App 中的 Looper/Handler 机制非常类似：App 端提交的图像数据并不会被 SurfaceFlinger 立即处理，而是先被放入一个缓冲队列中。只有当 SurfaceFlinger 进入合适的渲染时机（例如 VSYNC 信号触发时），才会从队列中取出这些数据进行合成。这样的设计可以有效地协调图像生产与消费的节奏，提升整体渲染效率和系统响应性能。

SurfaceFlinger Commit

SurfaceFlinger 的 commit 是什么时候被调用的呢？

答案是: 由 VSync 信号触发, 因为 SurfaceFlinger 注册了对 VSync 的监听，所以一旦 VSync 到来，会触发 Vsync 的回调，关于 VSync 部分，后面会有单独的一个章节描述，这里介绍回调的 flow

MessageQueue::vsyncCallback
    --> MessageQueue::Handler::dispatchFrame
        --> MessageQueue::Handler::handleMessage
            --> Scheduler::onFrameSignal

在 onFrameSignal 方法中会以此调用 SurfaceFlinger 的 commit 和 composite

void Scheduler::onFrameSignal(ICompositor& compositor, VsyncId vsyncId,
                              TimePoint expectedVsyncTime) {
    // 这里的 compositor 指的是 SurfaceFlinger
    ...
    compositor.commit(pacesetterPtr->displayId, targets);
    ...
    compositor.composite(pacesetterPtr->displayId, targeters);
    ...
}

我们接下来看 commit 是如何处理 App 传过来的 buffer

bool SurfaceFlinger::commit(PhysicalDisplayId pacesetterId,
                            const scheduler::FrameTargets& frameTargets) EXCLUDES(mStateLock) {

    ...
    // Composite if transactions were committed, or if requested by HWC.
    bool mustComposite = mMustComposite.exchange(false);
    {
        ...
        mustComposite |= updateLayerSnapshots(vsyncId, pacesetterFrameTarget.frameBeginTime().ns(),
                                              flushTransactions, transactionsAreEmpty);
        ...
}

继续分析 updateLayerSnapshots

bool SurfaceFlinger::updateLayerSnapshots(VsyncId vsyncId, nsecs_t frameTimeNs,
                                          bool flushTransactions, bool& outTransactionsAreEmpty) {
    using Changes = frontend::RequestedLayerState::Changes;
    SFTRACE_CALL();
    frontend::Update update;
    if (flushTransactions) {
        // 前面已经提到过 App 端提交的 Transaction 会保存到 mTransactionHandler 的队列中
        // 这里会收集所有的 Transactions
        mTransactionHandler.collectTransactions();
        {
            // TODO(b/238781169) lockless queue this and keep order.
            std::scoped_lock<std::mutex> lock(mCreatedLayersLock);
            update.legacyLayers = std::move(mCreatedLayers);
            mCreatedLayers.clear();
            // App 创建 Surface 的过程会在 SurfaceFlinger 创建 Layer
            // 创建的 Layer 都会添加到 mNewLayers 这个集合中
            update.newLayers = std::move(mNewLayers);
            mNewLayers.clear();
            update.layerCreationArgs = std::move(mNewLayerArgs);
            mNewLayerArgs.clear();
            update.destroyedHandles = std::move(mDestroyedHandles);
            mDestroyedHandles.clear();
        }
        // 把 mNewLayers 添加到 mLayerLifecycleManager 中
        mLayerLifecycleManager.addLayers(std::move(update.newLayers));
        update.transactions = mTransactionHandler.flushTransactions();
        if (mTransactionTracing) {
            mTransactionTracing->addCommittedTransactions(ftl::to_underlying(vsyncId), frameTimeNs,
                                                          update, mFrontEndDisplayInfos,
                                                          mFrontEndDisplayInfosChanged);
        }
        mLayerLifecycleManager.applyTransactions(update.transactions);
        mLayerLifecycleManager.onHandlesDestroyed(update.destroyedHandles);
        for (auto& legacyLayer : update.legacyLayers) {
            mLegacyLayers[legacyLayer->sequence] = legacyLayer;
        }
        mLayerHierarchyBuilder.update(mLayerLifecycleManager);
    }

    bool mustComposite = false;
    mustComposite |= applyAndCommitDisplayTransactionStatesLocked(update.transactions);

    {
        // 1.创建 LayerSnapshot
        frontend::LayerSnapshotBuilder::Args
                args{.root = mLayerHierarchyBuilder.getHierarchy(),
                     .layerLifecycleManager = mLayerLifecycleManager,
                     .displays = mFrontEndDisplayInfos,
                     .displayChanges = mFrontEndDisplayInfosChanged,
                     .globalShadowSettings = mDrawingState.globalShadowSettings,
                     .supportsBlur = mSupportsBlur,
                     ...
        mLayerSnapshotBuilder.update(args);
    }
    ...
    // 2.applyTransactionsLocked
    mustComposite |= applyTransactionsLocked(update.transactions);
    traverseLegacyLayers([&](Layer* layer) { layer->commitTransaction(); });
    const nsecs_t latchTime = systemTime();
    bool unused = false;
    // 遍历所有的 Layer
    for (auto& layer : mLayerLifecycleManager.getLayers()) {
        ...
        // 3.latchBufferImpl
        it->second->latchBufferImpl(unused, latchTime, bgColorOnly);
        
        ...
        // 4.添加 layer 到 mLayersWithQueuedFrames
        mLayersWithQueuedFrames.emplace(it->second, gameMode);
    }

    updateLayerHistory(latchTime);
    mLayerSnapshotBuilder.forEachSnapshot([&](const frontend::LayerSnapshot& snapshot) {
        if ((snapshot.isVisible && snapshot.contentDirty) ||
            (!snapshot.isVisible && snapshot.changes.test(Changes::Visibility))) {
            Region visibleReg;
            visibleReg.set(snapshot.transformedBoundsWithoutTransparentRegion);
            invalidateLayerStack(snapshot.outputFilter, visibleReg);
        }
    });
    ...

    return mustComposite;
}

1.创建 LayerSnapshot

在 mLayerSnapshotBuilder.update(args)中会调用 createSnapshot

LayerSnapshot* LayerSnapshotBuilder::createSnapshot(const LayerHierarchy::TraversalPath& path,
                                                    const RequestedLayerState& layer,
                                                    const LayerSnapshot& parentSnapshot) {
    
    mSnapshots.emplace_back(std::make_unique<LayerSnapshot>(layer, path));
    ...
    return snapshot;
}

在 createSnapshot 中创建了 LayerSnapshot (LayerSnapshot 作为 Layer 的"快照" 用来保持合成过程中的数据)并且把创建的 LayerSnapshot 添加到 mSnapshots 中，在后续的 Composition 会用到这个 mSnapshots

2.applyTransactionsLocked

bool SurfaceFlinger::applyTransactionsLocked(std::vector<QueuedTransactionState>& transactions)
        REQUIRES(mStateLock) {
    bool needsTraversal = false;
    // Now apply all transactions.
    for (auto& transaction : transactions) {
        needsTraversal |=
                applyTransactionState(transaction.frameTimelineInfo, transaction.states,
                                      transaction.displays, transaction.flags,
                                      transaction.inputWindowCommands,
                                      transaction.desiredPresentTime, transaction.isAutoTimestamp,
                                      std::move(transaction.uncacheBufferIds), transaction.postTime,
                                      transaction.hasListenerCallbacks,
                                      transaction.listenerCallbacks, transaction.originPid,
                                      transaction.originUid, transaction.id);
    }
    return needsTraversal;
}

在 applyTransactionsLocked 会遍历每个 Transaction，并调用 applyTransactionState

bool SurfaceFlinger::applyTransactionState(
        const FrameTimelineInfo& frameTimelineInfo, std::vector<ResolvedComposerState>& states,
        std::span<DisplayState> displays, uint32_t flags,
        const InputWindowCommands& inputWindowCommands, const int64_t desiredPresentTime,
        bool isAutoTimestamp, const std::vector<uint64_t>& uncacheBufferIds, const int64_t postTime,
        bool hasListenerCallbacks, const std::vector<ListenerCallbacks>& listenerCallbacks,
        int originPid, int originUid, uint64_t transactionId) REQUIRES(mStateLock) {
    ...
    for (auto& resolvedState : states) {
        clientStateFlags |=
                updateLayerCallbacksAndStats(frameTimelineInfo, resolvedState, desiredPresentTime,
                                             isAutoTimestamp, postTime, transactionId);
    }
    ...
}

继续看 updateLayerCallbacksAndStats

uint32_t SurfaceFlinger::updateLayerCallbacksAndStats(const FrameTimelineInfo& frameTimelineInfo,
                                                      ResolvedComposerState& composerState,
                                                      int64_t desiredPresentTime,
                                                      bool isAutoTimestamp, int64_t postTime,
                                                      uint64_t transactionId) REQUIRES(mStateLock) {
    // 从 ResolvedComposerState 拿到 layer_state_t 这个结构体
    layer_state_t& s = composerState.state;
    // layer_state_t.what 代表对 layer 的操作
    const uint64_t what = s.what;
    uint32_t flags = 0;
    sp<Layer> layer = nullptr;
    // 通过 layer_state_t.surface 拿到具体要操作的 Layer
    if (s.surface) {
        layer = LayerHandle::getLayer(s.surface);
    } else {
        // The client may provide us a null handle. Treat it as if the layer was removed.
        ALOGW("Attempt to set client state with a null layer handle");
    }
    ...
    // if buffer changed
    if (what & layer_state_t::eBufferChanged) {
        std::optional<ui::Transform::RotationFlags> transformHint = std::nullopt;
        if (snapshot) {
            transformHint = snapshot->transformHint;
        }
        layer->setTransformHint(transformHint);
        // 重点来了，终于在这里找到了 setBuffer
        if (layer->setBuffer(composerState.externalTexture, *s.bufferData, postTime,
                             desiredPresentTime, isAutoTimestamp, frameTimelineInfo, gameMode)) {
            flags |= eTraversalNeeded;
        }
        mLayersWithQueuedFrames.emplace(layer, gameMode);
    }
    ...
}   

1. 通过 layer_state_t.surface 获取到需要更新的 Layer，为什么能通过 surface 找到 Layer，上面已经提到了。
2. 通过 Layer 的 setBuffer 来更新 Buffer

bool Layer::setBuffer(std::shared_ptr<renderengine::ExternalTexture>& buffer,
                      const BufferData& bufferData, nsecs_t postTime, nsecs_t desiredPresentTime,
                      bool isAutoTimestamp, const FrameTimelineInfo& info, gui::GameMode gameMode) {
    ...
    把 ExternalTexture 转移给 DrawingState.buffer
    mDrawingState.buffer = std::move(buffer);
    ...
}

到这里为止 App 提交过来的 Buffer 就更新到了对应的 Layer, 保存到了 DrawingState 这个成员变量中。

3.latchBufferImpl

bool Layer::latchBufferImpl(bool& recomputeVisibleRegions, nsecs_t latchTime, bool bgColorOnly)
        REQUIRES(mFlinger->mStateLock) {
    ...
    gatherBufferInfo();
    ...
}

latchBufferImpl 的核心在 gatherBufferInfo

void Layer::gatherBufferInfo() {
    ...

    mBufferInfo.mBuffer = mDrawingState.buffer;
    ...
}

在前面 setBuffer 的部分，App 提交的 buffer 保存到了 Layer 的 mDrawingState.buffer 中，而在 latchBufferImpl中又把 mDrawingState.buffer 拷贝到 mBufferInfo.mBuffer 这个成员变量，而 mBufferInfo.mBuffer 将会参与后续 SurfaceFlinger 的 Composite。 这里解释一下，为什么要做 latch buffer 这个操作呢？我个人理解是为了状态隔离:

1. mDrawingState.buffer 是 App 提交的原始状态，可能在下一帧中被修改
2. mBufferInfo.mBuffer 是 SurfaceFlinger 当前帧合成所使用的 buffer，必须保持稳定。通过拷贝，可以确保当前帧的合成不会受到 App 后续事务的干扰。

4. 添加 layer 到 mLayersWithQueuedFrames

先看 mLayersWithQueuedFrames 的定义

std::unordered_set<std::pair<sp<Layer>, gui::GameMode>, LayerIntHash> mLayersWithQueuedFrames;

mLayersWithQueuedFrames 表示当前有新的 buffer 被 latch(锁定)到图层集合，这些 Layer 代表是活跃的，即 App 更新了 buffer 到这个 Layer 上。

到这里为止，App 通过 Transaction apply，成功把 Buffer 提交给了 SurfaceFlinger 的 Layer, 最终保存在了 mBufferInfo.mBuffer 这个成员变量中。在分析过程中，只抓住了主要矛盾：那就是 Buffer 从 App 到 SurfaceFlinger 的 layer 的 Transmission 过程。