unity3d:GameFramework+xLua+Protobuf+lua-protobuf,与服务器交互收发协议

概述

1.cs收发协议,通过protobuf序列化

2.lua收发协议,通过lua-protobuf序列化

一条协议字节流组成

C#协议基类

CSPacketBase,SCPacketBaseC#用协议基类

proto生成的CS类,基于这两个基类。分别为CSPacketBase是客户端发送至服务器,SCPacketBase是服务器发送至客户端

Q:为什么要区分这2个

A:反射注册所有SCPacketBase类,为C#接收协议反序列化候选

一个类示例

csharp 复制代码
  [global::System.Serializable, global::ProtoBuf.ProtoContract(Name=@"CSLogin")]
  public partial class CSLogin : CSPacketBase
  {
    public CSLogin() {}
    

    private string _account = "";
    [global::ProtoBuf.ProtoMember(1, IsRequired = false, Name=@"account", DataFormat = global::ProtoBuf.DataFormat.Default)]
    [global::System.ComponentModel.DefaultValue("")]
    public string account
    {
      get { return _account; }
      set { _account = value; }
    }

    private string _password = "";
    [global::ProtoBuf.ProtoMember(2, IsRequired = false, Name=@"password", DataFormat = global::ProtoBuf.DataFormat.Default)]
    [global::System.ComponentModel.DefaultValue("")]
    public string password
    {
      get { return _password; }
      set { _password = value; }
    } 
	//网络协议Id
	public override int Id 
	{ 
		get {return (int)Network.NetMsgID.CSLogin;} 
	} 

	//回到引用池,变量设置初始化。如果是引用型成员变量也要回到引用池
	public override void Clear()
	{
		_account = "" ;
		_password = "" ;

	}
     	
  }

SCPacketLua

C#中接收包,传递给Lua处理

其中m_id为协议id,m_len为字节数组长度

csharp 复制代码
public class SCPacketLua : PacketBase
{
    public int m_id;
    public int m_len;
    public PacketBuffer m_bytes; 

在网络事件使用完毕GameFramework.EventPool.HandleEvent 中

csharp 复制代码
ReferencePool.Release(e);

触发回收

csharp 复制代码
    public override void Clear()
    {
        ReferencePool.Release(m_bytes);
        m_bytes = null; //要去引用,不然引用池那释放不了 

CSPacketLua

C#中用于接收从lua传递过来的字节流,发送给服务器

csharp 复制代码
    public  class CSPacketBaseLua : PacketBase
    {
        public PacketBuffer m_bytes; //发送字节流
        public int m_id = 0; //协议id
        public ushort Len //字节流长度

C#中发送协议

csharp 复制代码
CSLogin csLogin = ReferencePool.Acquire<CSLogin>();
csLogin.account = "123";
csLogin.password = "456";
GameEntry.Network.Send(csLogin);

主线程遍历发送队列

每有一个发送,把packet放入到发送队列中,unity主线程中遍历发送队列,把当前帧的全部待发送包,序列化到一个流中

GameFramework.Network.NetworkManager.NetworkChannelBase.ProcessSend

csharp 复制代码
if (m_SendState.Stream.Length > 0 || m_SendPacketPool.Count <= 0)
{
    //发送流中还有未发送完的或者没有待发送的包
    return false;
}
//所有在发送队列中,都序列化到流中
while (m_SendPacketPool.Count > 0)
{
    Packet packet = null;
    lock (m_SendPacketPool)
    {
        //每次从发送队列中取一个
        packet = m_SendPacketPool.Dequeue();
    }

    bool serializeResult = false;
    try
    {
        serializeResult = m_NetworkChannelHelper.Serialize(packet, m_SendState.Stream);
    }
}
//流的操作,写完,要回到起点
m_SendState.Stream.Position = 0L;    

序列化消息包

StarForce.NetworkChannelHelper.Serialize

csharp 复制代码
/// <summary>
/// 序列化消息包。
/// </summary>
/// <typeparam name="T">消息包类型。</typeparam>
/// <param name="packet">要序列化的消息包。</param>
/// <param name="destination">要序列化的目标流。</param>
/// <returns>是否序列化成功。</returns>
public bool Serialize<T>(T packet, Stream destination) where T : Packet
{
    //todo:频繁is as 会有性能消耗
    if (packet is CSPacketBase)
    {
        PacketBase packetImpl = packet as CSPacketBase;
        //先写入body
        m_CachedStream.SetLength(1024*8); // 此行防止 Array.Copy 的数据无法写入
        m_CachedStream.Position = PacketHeaderLength;
        Serializer.Serialize(m_CachedStream, packet);

        //获得body长度,再写入消息头,id,body长度
        ushort bodyLen = (ushort)((int)m_CachedStream.Position - PacketHeaderLength);

        arrID = BitConverter.GetBytes(packetImpl.Id);
        arrBodyLen = BitConverter.GetBytes(bodyLen);
        m_CachedStream.Position = 0;
        m_CachedStream.Write(arrID, 0, 4);
        m_CachedStream.Write(arrBodyLen, 0, 2);


        m_CachedStream.SetLength(PacketHeaderLength + bodyLen);
        byte[] arrBytes = m_CachedStream.ToArray();
        Log.Info("序列化字节流:{0}", BitConverter.ToString(arrBytes));
        m_CachedStream.WriteTo(destination);
        ReferencePool.Release(packet);
        return true;
    }
    //else if (packetImpl.PacketType == PacketType.ClientToServerLua)
    else if (packet is CSPacketLua)
    {
        m_CachedStream.SetLength(1024 * 8); // 此行防止 Array.Copy 的数据无法写入
        CSPacketLua packetLua = packet as CSPacketLua;  
        arrID = BitConverter.GetBytes(packetLua.Id);
        arrBodyLen = BitConverter.GetBytes(packetLua.Len);
        m_CachedStream.Position = 0; //每次开始写流,流位置要设置为0,代表起始位置。每次写byte,pos会自动增加
        m_CachedStream.Write(arrID, 0, 4);
        m_CachedStream.Write(arrBodyLen, 0, 2);
        m_CachedStream.Write(packetLua.m_bytes.Buffer, 0, packetLua.Len);
        m_CachedStream.SetLength(PacketHeaderLength + packetLua.Len);//写完流,要设置下流的真实长度。因为流是复用的,不然不会截断
        byte[] arrBytes = m_CachedStream.ToArray();
        Log.Info("序列化字节流Lua:{0}", BitConverter.ToString(arrBytes));
        m_CachedStream.WriteTo(destination);//缓存流写入到发送流中
        ReferencePool.Release(packet);
        return true;
    }

    Log.Warning("Send packet invalid.");
    return false;
}

对于CSPacketBase类型

1.m_CachedStream是每个packet序列化的流,每次使用前需要设置position

2.先设置m_CachedStream.Position = PacketHeaderLength; 先跳过id,bodyLen位置,先写入body

3.protobuf序列化Serializer.Serialize(m_CachedStream, packet);后得到ushort bodyLen = (ushort)((int)m_CachedStream.Position - PacketHeaderLength);即为bodyLen长度

4.再设置m_CachedStream.Position = 0;,写入id字节流,bodyLen字节流

5.m_CachedStream.SetLength(PacketHeaderLength + bodyLen);写完后要设置长度截断,因为m_CachedStream是复用的,可能上次使用后面还有字节数据

  1. m_CachedStream.WriteTo(destination);即为发送流,每次会添加到发送流的末尾

对于CSPacketLua类型

1.由于byte是在lua中序列化好的传递到C#的,只需要按照顺序写入到m_CachedStream中,其他流程与CSPacketBase一致

发送流

GameFramework.Network.NetworkManager.TcpWithSyncReceiveNetworkChannel.SendAsync

csharp 复制代码
private void SendAsync()
{
    try
    {
        m_Socket.BeginSend(m_SendState.Stream.GetBuffer(), (int)m_SendState.Stream.Position, (int)(m_SendState.Stream.Length - m_SendState.Stream.Position), SocketFlags.None, m_SendCallback, m_Socket);
    }

每次从流中取position到length部分。每次发送一个完整流,从0开始。进入到发送回调

GameFramework.Network.NetworkManager.TcpWithSyncReceiveNetworkChannel.SendCallback

csharp 复制代码
private void SendCallback(IAsyncResult ar)
{
    Socket socket = (Socket)ar.AsyncState;
   int bytesSent = 0;
    try
    {
        bytesSent = socket.EndSend(ar);
    }

    m_SendState.Stream.Position += bytesSent;
    if (m_SendState.Stream.Position < m_SendState.Stream.Length)
    {
        SendAsync();
        return;
    }

发送了一段,设置流位置

如果位置<Length,接着调用发送,直到把流全部发送完毕

C#中接收协议

初始化时反射注册协议id对应type,协议id对应处理Handle

StarForce.NetworkChannelHelper.Initialize

csharp 复制代码
// 反射注册包和包处理函数。
Type packetBaseType = typeof(SCPacketBase);
Type packetHandlerBaseType = typeof(PacketHandlerBase);
Assembly assembly = Assembly.GetExecutingAssembly();
Type[] types = assembly.GetTypes();
for (int i = 0; i < types.Length; i++)
{
    if (!types[i].IsClass || types[i].IsAbstract)
    {
        continue;
    }

    if (types[i].BaseType == packetBaseType)
    {
        //确定msgID反序列化的类结构
        PacketBase packetBase = (PacketBase)Activator.CreateInstance(types[i]);
        Type packetType = GetServerToClientPacketType(packetBase.Id);
        //防止监听的协议id重复
        if (packetType != null)
        {
            Log.Warning("Already exist packet type '{0}', check '{1}' or '{2}'?.", packetBase.Id.ToString(), packetType.Name, packetBase.GetType().Name);
            continue;
        }

        m_ServerToClientPacketTypes.Add(packetBase.Id, types[i]);
    }
    else if (types[i].BaseType == packetHandlerBaseType)
    {
        //网络事件handle处理
        IPacketHandler packetHandler = (IPacketHandler)Activator.CreateInstance(types[i]);
        m_NetworkChannel.RegisterHandler(packetHandler);
    }
}

异步接收流

连接成功,开始异步接收流

GameFramework.Network.NetworkManager.TcpNetworkChannel.ReceiveAsync

csharp 复制代码
//每次获取到完整头/body。流会pos = 0,len为头长,或者body长
//每次拆包都是读取pos累积,剩余要读的为len-pos
m_Socket.BeginReceive(m_ReceiveState.Stream.GetBuffer(), (int)m_ReceiveState.Stream.Position, (int)(m_ReceiveState.Stream.Length - m_ReceiveState.Stream.Position), SocketFlags.None, m_ReceiveCallback, m_Socket);

有服务器下发协议,进入到接收回调中

GameFramework.Network.NetworkManager.TcpNetworkChannel.ReceiveCallback

csharp 复制代码
private void ReceiveCallback(IAsyncResult ar)
{
    Socket socket = (Socket)ar.AsyncState;
  
    int bytesReceived = 0;
    try
    {
        bytesReceived = socket.EndReceive(ar);
    }
    //每次读取pos会累加
    m_ReceiveState.Stream.Position += bytesReceived;
    if (m_ReceiveState.Stream.Position < m_ReceiveState.Stream.Length)
    {
        //未读满上次设定长度,接着读。上次设定长度为HeadLen,BodyLen
        ReceiveAsync();
        return;
    }
    //开始解析,流要置为0位置开始解析
    m_ReceiveState.Stream.Position = 0L;

    bool processSuccess = false;
    if (m_ReceiveState.PacketHeader != null)
    {
        //上次获取到头,这次反序列处理body
        processSuccess = ProcessPacket();
        m_ReceivedPacketCount++;
    }
    else
    {
        //未获取到头,反序列化头
        processSuccess = ProcessPacketHeader();
    }

    if (processSuccess)
    {
        //处理成功,接着接收
        ReceiveAsync();
        return;
    }
}

1.初始化时,设置第一次流接收Length为6(协议id int+ bodyLen ushort)。即待接收包头

2.如果接收满了Length,进入到处理包头,解析出协议id,bodyLen

3.设置下一次接收为Length为bodyLen。即待接收包体

4.如果接收满Length,此时进入到处理包体,解析出body对应的对象。设置下次接收为包头Length6,循环到第一步

注意事项

如果有拆包黏包,在接收回调中处理,并且接满一个模式,再解析

csharp 复制代码
    m_ReceiveState.Stream.Position += bytesReceived;
    if (m_ReceiveState.Stream.Position < m_ReceiveState.Stream.Length)
    {
        //未读满上次设定长度,接着读。上次设定长度为HeadLen,BodyLen
        ReceiveAsync();
        return;
    }

每次开始解析前,需要流postion = 0开始,因为随着接收,position到了末尾,无法解析

csharp 复制代码
    //开始解析,流要置为0位置开始解析
    m_ReceiveState.Stream.Position = 0L;

解析包头

StarForce.NetworkChannelHelper.DeserializePacketHeader

csharp 复制代码
source.Position = 0;
SCPacketHeader scHead = ReferencePool.Acquire<SCPacketHeader>();
source.Read(arrID, 0, 4);
scHead.Id = BitConverter.ToInt32(arrID,0);
source.Read(arrBodyLen, 0, 2);
scHead.PacketLength = BitConverter.ToUInt16(arrBodyLen, 0);

得到协议id,bodyLen

解析包体

StarForce.NetworkChannelHelper.DeserializePacket

csharp 复制代码
Packet packet = null;
if (scPacketHeader.IsValid)
{
    Type packetType = GetServerToClientPacketType(scPacketHeader.Id);
    if (packetType != null)
    {
        //source 为接收流,每次接收一整条消息前,设置了Len
        packet = (Packet)ReferencePool.Acquire(packetType);
        packet = (Packet)RuntimeTypeModel.Default.Deserialize(source, packet, packetType);
    }
    else
    {
        //如果id找不到字节流传递给Lua中反序列化为table
        SCPacketLua luaPacket = ReferencePool.Acquire<SCPacketLua>();//引用池中使用,后续会在使用事件通知后,回到引用池
        luaPacket.m_id = scPacketHeader.Id;
        luaPacket.m_len = scPacketHeader.PacketLength;
        luaPacket.m_bytes = PacketBuffer.GetBuffer(scPacketHeader.PacketLength);
        source.Read(luaPacket.m_bytes.Buffer, 0, scPacketHeader.PacketLength);
        packet = luaPacket;

1.根据包体id(协议id),找到初始化反射注册的协议id,type

2.如果有,说明是C#用协议,protobuf反序列化为对象,加入到事件队列中,等待分发,这样做事为了从其他线程中转回主线程处理

3.如果不存在type,说明是Lua用协议,把字节流保存到SCPacketLua,传递到Lua处理字节流转为Lua中table

网络消息分发

GameFramework.EventPool.HandleEvent

csharp 复制代码
if (m_EventHandlers.TryGetValue(e.Id, out range))
{
    LinkedListNode<EventHandler<T>> current = range.First;
    while (current != null && current != range.Terminal)
    {
        m_CachedNodes[e] = current.Next != range.Terminal ? current.Next : null;

        if (m_NoSenderDict.ContainsKey(e.Id) && m_NoSenderDict[e.Id] == current.Value)
        {
            current.Value(null, e);
        }
        else
        {
            //这里会传递网络handle
            current.Value(sender, e);
        }

        current = m_CachedNodes[e];
    }

    m_CachedNodes.Remove(e);
}
else if (m_DefaultHandler != null)
{
    m_DefaultHandler(sender, e);
}
else if ((m_EventPoolMode & EventPoolMode.AllowNoHandler) == 0)
{
    noHandlerException = true;
}

ReferencePool.Release(e);

1.C#中初始化时反射注册协议id对应handle,例如handle中内容

csharp 复制代码
public class SCLoginHandler : PacketHandlerBase
{

    public override int Id
    {
        get
        {
            return (int)Network.NetMsgID.SCLogin;
        }
    }

    public override void Handle(object sender, Packet packet)
    {
        SCLogin packetImpl = (SCLogin)packet;
        Log.Info("SCLoginHandler name:{0}-passeword{1}", packetImpl.account, packetImpl.password);
    }
}

会在current.Value(sender, e);中调用到 Handle,这里可以把协议转换好的对象,进一步处理

2.未找到协议id对应handle,执行m_DefaultHandler(sender, e);,这里可以在初始化设置委托在lua中执行,把SCPacketLua传递到Lua进一步处理

Lua中发送协议

lua中

csharp 复制代码
function TestSendPlayerInfo()
    --每次协议都是全手写table,未看到可生成协议格式.lua文件,这样不可复用,每次都需要看proto描述,写全部字段。
    --需要在业务module中手写一遍全部协议send函数,确定发送的参数,组装成一个table再发送。这样相当于手动写了一遍
    local data = {
        name = "789",
        level = 123
    }
    LuaEntry.NetworkModule:Send(NetMsgID.CSPlayerInfo,data)
end

function NetworkModule:Send(msgID,data)
    local sProto = MsgID2Proto[msgID]
    if sProto == nil then
        Log.Info("消息ID:{0}找不到需要序列化的proto对象",msgID)
        return
    end
    local bytes = assert(pb.encode(sProto, data))--返回值虽然为string,但是这是字节数组在lua中表达,可以直接传递给C#的byte[]
    local sHex = pb.tohex(bytes)
    Log.Info("Send Hex:{0}",sHex)
    GameEntry.Network:SendByLua(msgID,bytes) --有时候调用不到,生成一次wrap,再清除掉
end

1.lua中声明一个table为packet,里面字段为proto中描述

2.assert(pb.encode(sProto, data),table序列化二进制数组,返回值虽然为string,但这是字节数组在lua中的表达,可以直接传递到c#的byte[]中https://www.jianshu.com/p/63987134c1ba

C#处理

csharp 复制代码
public static void SendByLua(this NetworkComponent networkComponent, int msgID, byte[] bytes)
{
    CSPacketLua packet = ReferencePool.Acquire<CSPacketLua>();
    packet.m_id = msgID;
    packet.m_bytes = PacketBuffer.GetBufferAndCopyBytes(bytes,bytes.Length);
    packet.Len = (ushort)bytes.Length;
    networkComponent.Send<CSPacketLua>(packet);
}

Lua中接收协议

C#中注册网络委托

csharp 复制代码
public static Action<SCPacketLua> CreateFunc = null;
m_NetworkChannel.SetDefaultHandler(LuaPacketHandler);

在网络消息分发时,未找打id对应handle,再进入到网络委托中处理

GameFramework.EventPool.HandleEvent

csharp 复制代码
 m_DefaultHandler(sender, e);

Lua中执行网络委托内容

csharp 复制代码
SF.NetworkChannelHelper.CreateFunc = handler(self,self.CreatePacket)

function NetworkModule:CreatePacket(packet)
    local luaPacket = packet
    local sProto = MsgID2Proto[luaPacket.Id]
    if sProto == nil then
        Log.Info("消息ID:{0}找不到需要序列化的proto对象",luaPacket.Id)
        return
    end
    local data = assert(pb.decode(sProto, luaPacket.m_bytes.Buffer))
    PrintTable(data,false,true,"CreatePacket")
end

把C#中传递过来的SCPacketLua中的字节流使用lua-protobuf反序列化为table

流程图

GFxLuaProto发送协议流程图

GFxLuaProto接收协议流程图

遇到错误

字节流长度不对

ProtoBuf.ProtoException: Invalid wire-type; this usually means you have over-written a file without truncating or setting the length

使用字节流反序列化错误,检查长度之类

复用流,每次使用完要进行截断SetLength,否则会带入上次长度

流每次写入,都会改变position位置

lua-protobuf中反序列化,默认值问题

如果protobuf的成员值为默认值,序列化后会缺省这部分字节流。

lua中反序列化不出这个member。需要设置lua-protobuf中使用默认值

csharp 复制代码
pb.option "use_default_values" --将默认值表复制到解码目标表中来

安卓测试

从C#发送,C#接收处打印

从Lua发送,Lua接收处打印

相关推荐
Jtti几秒前
Windows系统服务器怎么设置远程连接?详细步骤
运维·服务器·windows
yeyuningzi14 分钟前
Debian 12环境里部署nginx步骤记录
linux·运维·服务器
EasyCVR1 小时前
萤石设备视频接入平台EasyCVR多品牌摄像机视频平台海康ehome平台(ISUP)接入EasyCVR不在线如何排查?
运维·服务器·网络·人工智能·ffmpeg·音视频
龙哥说跨境3 小时前
如何利用指纹浏览器爬虫绕过Cloudflare的防护?
服务器·网络·python·网络爬虫
pk_xz1234564 小时前
Shell 脚本中变量和字符串的入门介绍
linux·运维·服务器
小珑也要变强4 小时前
Linux之sed命令详解
linux·运维·服务器
海绵波波1074 小时前
Webserver(4.3)TCP通信实现
服务器·网络·tcp/ip
九河云6 小时前
AWS账号注册费用详解:新用户是否需要付费?
服务器·云计算·aws
Lary_Rock6 小时前
RK3576 LINUX RKNN SDK 测试
linux·运维·服务器
幺零九零零7 小时前
【计算机网络】TCP协议面试常考(一)
服务器·tcp/ip·计算机网络