Java在网络通信中应该如何选择合适的序列化框架?

前言

说到网络通信就会牵涉到数据的序列化与反序列化，现如今序列化框架也是层出不穷，比如FST、Kryo、ProtoBuffer、Thrift、Hessian、Avro、MsgPack等等，有的人可能会有疑问，为什么市面上有这么多框架，JDK不是已经有自带的Serializable序列化接口吗?很遗憾地说出这个事实，作为JDK自带地序列化机制，无论是在时间还是空间上的性能不尽人意，但凡时间或者空间上性能优越一点，也不至于让人诟病这么久。所以也就出现了这么多序列化框架，另外即便JDK序列化可以实现，但是无法在跨语言的网络通信中表现出色，除非在多个语言端各自定义好每一个对象，但是这样做无疑效率是最低的，需要提前定义的对象太多了，人力明显不够用。所以也就出现了以ProtoBuffer、Kryo、Thrift等支持跨语言的框架，接下来让我们看下各个框架之间的比较，有比较才能在业务中有更多的选择

序列化框架	通用性
JDK Serializer	只适用于Java
FST	只适用于Java
Kryo	主要适用于Java(可复杂支持多种语言)
Protocol buffer	支持多种语言
Thrift	支持多种语言
Hessian	支持多种语言
Avro	支持多种语言
MsgPack	支持多种语言

性能比较

从时间和空间复杂度上来说，kryo的表现是其他序列化框架当中比较好的，序列化之后的大小空间上以及序列化/反序列化时间都不错，但是它没有那么直接不支持跨语言。

接下来我们再来看看针对大小数据序列化的结果，数据的大小也会影响序列化的结果。

分析结果

如果你的系统架构设计中设计到了多语言，那么Proto Buffer和MsgPack，avro将会是个不错的选择，当你的系统是Java时，还可以考虑kryo，它的序列化和反序列化时间相对均衡。

使用示例

    <dependencies>
        <dependency>
            <groupId>io.netty</groupId>
            <artifactId>netty-all</artifactId>
            <version>4.1.42.Final</version>
        </dependency>

        <dependency>
            <groupId>org.msgpack</groupId>
            <artifactId>msgpack</artifactId>
            <version>0.6.12</version>
        </dependency>

        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>1.7.30</version>
            <scope>compile</scope>
        </dependency>

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-core</artifactId>
            <version>1.2.4</version>
        </dependency>

        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-classic</artifactId>
            <version>1.2.4</version>
        </dependency>

        <dependency>
            <groupId>com.itextpdf</groupId>
            <artifactId>itextpdf</artifactId>
            <version>5.5.8</version>
        </dependency>
        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcprov-jdk15on</artifactId>
            <version>1.49</version>
            <type>jar</type>
            <scope>compile</scope>
            <optional>true</optional>
        </dependency>
        <dependency>
            <groupId>org.bouncycastle</groupId>
            <artifactId>bcpkix-jdk15on</artifactId>
            <version>1.49</version>
            <type>jar</type>
            <scope>compile</scope>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>de.javakaffee</groupId>
            <artifactId>kryo-serializers</artifactId>
            <version>0.42</version>
        </dependency>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13.1</version>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.google.protobuf</groupId>
            <artifactId>protobuf-java</artifactId>
            <version>2.6.1</version>
        </dependency>
    </dependencies>

1.ProtoBuffer

Proto文件如何生成JavaProto对象?这篇博客已经介绍过了ProtoBuffer了，这里不再赘述，有兴趣的小伙伴可以看看

2.Kryo

实体类

package serializable.protogenesis;

import org.msgpack.annotation.Message;

import java.io.Serializable;
import java.nio.ByteBuffer;

@Message
public class UserInfo implements Serializable {
    /**
     * 默认序列号
     */
    private static final long serialVersionUID = 7627113094707995002L;
    
    private String userName;
    
    private int userID;

    public String getUserName() {
        return userName;
    }

    public UserInfo setUserName(String userName) {
        this.userName = userName;
        return this;
    }

    public int getUserID() {
        return userID;
    }

    public UserInfo setUserID(int userID) {
        this.userID = userID;
        return this;
    }
    
    // 自行序列化
    public byte[] codeC() {
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        // userName转换为字节数组value
        byte[] value = this.userName.getBytes();
        // 写入字节数组value的长度
        buffer.putInt(value.length);
        // 写入字节数组value的值
        buffer.put(value);
        // 写入userID的值
        buffer.putInt(this.userID);
        // 准备读取buffer中的数据
        buffer.flip();
        value = null;
        byte[] result = new byte[buffer.remaining()];
        // buffer中的数据写入字节数组并作为结果返回
        buffer.get(result);
        return result;
        
    }
    
    // 自行序列化方法2
    public byte[] codeC(ByteBuffer buffer) {
        buffer.clear();
        byte[] value = this.userName.getBytes();
        buffer.putInt(value.length);
        buffer.put(value);
        buffer.putInt(this.userID);
        buffer.flip();
        value = null;
        byte[] result = new byte[buffer.remaining()];
        buffer.get(result);
        return result;
    }


    @Override
    public String toString() {
        return "UserInfo{" +
                "userName='" + userName + '\'' +
                ", userID=" + userID +
                '}';
    }
}

KryoSerializer

package adv.kryocodec;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.io.Input;
import com.esotericsoftware.kryo.io.Output;
import io.netty.buffer.ByteBuf;
import io.netty.buffer.ByteBufInputStream;

import java.io.ByteArrayOutputStream;
import java.io.IOException;

public class KryoSerializer {
    
    private static Kryo kryo = KryoFactory.createKryo();
    
    // 序列化
    public static void serialize(Object object, ByteBuf out) {
        long start = System.currentTimeMillis();
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        Output output = new Output(baos);
        kryo.writeClassAndObject(output, object);
        
        output.flush();
        output.close();

        byte[] b = baos.toByteArray();
        try {
            baos.flush();
            baos.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        
        out.writeBytes(b);

        long end = System.currentTimeMillis();
        System.out.println("The Kryo serializable length is "+ b.length +", serialize time is :" + (end - start));
    }
    
    // 序列化为一个字节数组，主要用在消息摘要上
    public static byte[] obj2Bytes(Object object) {
        ByteArrayOutputStream baos = new ByteArrayOutputStream();
        Output output = new Output(baos);
        kryo.writeClassAndObject(output, object);
        output.flush();
        output.close();

        byte[] b = baos.toByteArray();
        try {
            baos.flush();
            baos.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
        
        return b;
    }
    
    
    public static Object deserialize(ByteBuf out) {
        if (out == null) {
            return null;
        }
        
        Input input = new Input(new ByteBufInputStream(out));
        return kryo.readClassAndObject(input);
    }
}

KryoFactory

在这个工厂中我们提前注册了很多序列化器，这样会加快我们序列化的速度

package adv.kryocodec;

import com.esotericsoftware.kryo.Kryo;
import com.esotericsoftware.kryo.serializers.DefaultSerializers;
import de.javakaffee.kryoserializers.*;

import java.lang.reflect.InvocationHandler;
import java.math.BigDecimal;
import java.math.BigInteger;
import java.net.URI;
import java.text.SimpleDateFormat;
import java.util.*;
import java.util.concurrent.ConcurrentHashMap;
import java.util.regex.Pattern;

public class KryoFactory {
    
    
    public static Kryo createKryo() {
        Kryo kryo = new Kryo();
        kryo.setRegistrationRequired(false);
        kryo.register(Arrays.asList("").getClass(),  new ArraysAsListSerializer());
        kryo.register(GregorianCalendar.class, new GregorianCalendarSerializer());
        kryo.register(InvocationHandler.class, new JdkProxySerializer());
        kryo.register(BigDecimal.class, new DefaultSerializers.BigDecimalSerializer());
        kryo.register(BigInteger.class, new DefaultSerializers.BigIntegerSerializer());
        kryo.register(Pattern.class, new RegexSerializer());
        kryo.register(BitSet.class, new BitSetSerializer());
        kryo.register(URI.class, new URISerializer());
        kryo.register(UUID.class, new UUIDSerializer());
        
        UnmodifiableCollectionsSerializer.registerSerializers(kryo);
        SynchronizedCollectionsSerializer.registerSerializers(kryo);
        
        kryo.register(HashMap.class);
        kryo.register(ArrayList.class);
        kryo.register(LinkedList.class);
        kryo.register(HashSet.class);
        kryo.register(TreeSet.class);
        kryo.register(Hashtable.class);
        kryo.register(Date.class);
        kryo.register(Calendar.class);
        kryo.register(ConcurrentHashMap.class);
        kryo.register(SimpleDateFormat.class);
        kryo.register(GregorianCalendar.class);
        kryo.register(Vector.class);
        kryo.register(BitSet.class);
        kryo.register(StringBuffer.class);
        kryo.register(StringBuilder.class);
        
        kryo.register(Object.class);
        kryo.register(Object[].class);
        kryo.register(String[].class);
        kryo.register(byte[].class);
        kryo.register(char[].class);
        kryo.register(int[].class);
        kryo.register(float[].class);
        kryo.register(double[].class);
        
        return kryo;
    }
}

TestPerform

package adv.kryocodec;

import io.netty.buffer.ByteBuf;
import io.netty.buffer.Unpooled;
import serializable.protogenesis.UserInfo;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;

public class TestPerform {

    public static void main(String[] args) throws IOException {
        UserInfo info = new UserInfo();
        info.setUserID(100).setUserName("Hello World");
        long start = System.currentTimeMillis();
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream os = new ObjectOutputStream(bos);
        os.writeObject(info);
        os.flush();
        os.close();

        byte[] b = bos.toByteArray();
        long end = System.currentTimeMillis();
        System.out.println("The JDK serializable length is :" + b.length + ", time is :" + (end - start));
        ByteBuf sendBuf = Unpooled.buffer();
        KryoSerializer.serialize(info, sendBuf);
        UserInfo deserialize = (UserInfo)KryoSerializer.deserialize(sendBuf);
        System.out.println("deserialize = " + deserialize.toString());
    }
}

3.MsgPack

TestMsgPackPerform

package serializable.msgpack;

import org.msgpack.MessagePack;
import serializable.protogenesis.UserInfo;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;

public class TestMsgPackPerform {

    public static void main(String[] args) throws IOException {
        UserInfo info = new UserInfo();
        info.setUserID(100).setUserName("Hello World");
        long start = System.currentTimeMillis();
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream os = new ObjectOutputStream(bos);
        os.writeObject(info);
        os.flush();
        os.close();

        byte[] b = bos.toByteArray();
        long end = System.currentTimeMillis();
        System.out.println("The JDK serializable length is :" + b.length + ", time is :" + (end - start));

        
        MessagePack messagePack = new MessagePack();
        byte[] bytes = messagePack.write(info);

        UserInfo read = messagePack.read(bytes, UserInfo.class);
        System.out.println("The MsgPack serializable length is :" + bytes.length);
        System.out.println("read = " + read);
    }
}

由于在序列化时会申请ByteBuf来操作，所以这个申请内存(无论是堆内存还是直接内存)都是需要耗费点时间的，所以不能把这个时间也算进去，这样我们得到的时间是不准的,只要在大量的序列化情况下才能看出效果。另外注意在实体类上加@Message这个注解，否则会报错

附录：NIO序列化

我们都知道NIO中也有ByteBuffer可以用来做序列化

TestUserInfo

public class TestUserInfo {

    public static void main(String[] args) throws IOException {
        UserInfo info = new UserInfo();
        info.setUserID(100).setUserName("Hello World");
        ByteArrayOutputStream bos = new ByteArrayOutputStream();
        ObjectOutputStream os = new ObjectOutputStream(bos);
        os.writeObject(info);
        os.flush();
        os.close();

        byte[] b = bos.toByteArray();
        System.out.println("The JDK serializable length is :" + b.length);
        bos.close();
        System.out.println("-------------------------------------------");
        System.out.println("the byte array serializable length is :" + info.codeC().length);

    }
}

PerformTestUserInfo

package serializable.protogenesis;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.ObjectOutputStream;
import java.nio.ByteBuffer;

public class PerformTestUserInfo {

    public static void main(String[] args) throws IOException {
        UserInfo info = new UserInfo();
        info.setUserID(100).setUserName("Hello World");
        int loop = 1000000;

        ByteArrayOutputStream bos = null;
        ObjectOutputStream os = null;

        long startTime = System.currentTimeMillis();
        for (int i = 0; i < loop; i++) {
            
            bos = new ByteArrayOutputStream();
            os = new ObjectOutputStream(bos);
            os.writeObject(info);
            os.flush();
            os.close();
            byte[] bytes = bos.toByteArray();
            bos.close();
        }
        long endTime = System.currentTimeMillis();
        System.out.println("The JDK serializable cost time is :" + (endTime - startTime) + "ms");
        System.out.println("---------------------------------");
        
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        startTime = System.currentTimeMillis();
        for (int i = 0; i < loop; i++) {
            byte[] bytes = info.codeC(buffer);
        }
        endTime = System.currentTimeMillis();

        System.out.println("The byte array serializable cost time is :" + (endTime - startTime) + "ms");
        

    }
}

NIO原生的序列化机制也有很不错的机制