learn - 技术栈

版本控制

我们使用版本控制来管理数据变更

推送 PUSH

拉回 PULL

Git是一款分布式版本控制系统

SVN是集中式版本控制系统

有什么区别？

集中式：

连线工作：提交(commit)、查看日志(log)、比较差异(diff)等核心操作需要网络连接。

提交即生效： svn commit 会直接将修改同步中央服务器，影响所有人。

单一中央仓库：所有版本历史都保存在唯一的中央服务器上。

分支昂贵：创建分支通常在服务器端完成，本质是在服务器目录中做一次拷贝，过程较慢，管理成本高。

我还以为git也是只有一个中心仓库呢？

AI:

99%的团队确实只用一个中心仓库。但这并非Git强制要求，而是团队为了方便管理而主动约定的"规矩"。

实际上，系统允许你从任何地方拉代码，但为了不混乱，大家约定俗成只认一个仓库为中心仓库。

Git:

当服务器宕机时，你可以把同事电脑上的仓库设为临时中心，继续合并代码，等服务器恢复后再同步过去。

Git给了你"去中心化"的能力，但为了效率，大家自愿选择了"中心化"的用法。

diff

diff是Linux/Unix系统下最经典的文本差异比较命令行工具，主要用于逐行比较两个文件或目录的源码差异。

diff -y main.cpp main1.cpp 并排显示(感觉蛮好用的）

FPGA

是否可以同时使用tcp和udp？

可以，FPGA完全能同时处理TCP和UDP协议，这是网络通信设计的常规操作。

强烈不推荐在FPGA逻辑中纯硬件实现FTP客户端。
可能需要学习下FPGA

TCP通信

Server

C++ 复制代码

#include <QCoreApplication>
#include <QTcpServer>
#include <QTcpSocket>
#include <QFile>
int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QTcpServer server;
    if(!server.listen(QHostAddress::Any,6000))
    {
        qDebug() << __FILE__ << __LINE__ << "Server could not start!";
    }
    QObject::connect(&server,&QTcpServer::newConnection,[&](){
       QTcpSocket * clientSocket = server.nextPendingConnection();
       qDebug() << clientSocket->peerAddress().toString()
                << clientSocket->peerPort();
       clientSocket->write("Hello from Tcp Server");

       QObject::connect(clientSocket,&QTcpSocket::readyRead,[clientSocket](){
           QByteArray data = clientSocket->readAll();
           qDebug() << "rec:" << data;

           QString response = "Echo:" + QString::fromUtf8(data).trimmed() + "\r\n";
           clientSocket->write(response.toUtf8());
       });
       QObject::connect(clientSocket,&QTcpSocket::disconnected,[clientSocket](){
           qDebug() << "Client disconnected: " << clientSocket->peerAddress().toString();
           clientSocket->deleteLater();
       });
    });
    return a.exec();
}

1.监听指定端口

2.接受客户端连接

3.接收客户端消息

4.回显消息给客户端

5.自动清理断开的连接

QTcpServer : TCP服务器

QTcpSocket : 客户端连接套接字

newConnection : 新客户端连接

readyRead : 有数据可读

disconnected : 客户端断开

Client

客户端测试:

telnet 127.0.0.1 6000

Telnet

Telnet是一个基于TCP/IP协议的远程登录工具，本质是一个纯文本的TCP客户端。

它不关心数据内容，只负责建立连接和收发字节流。

它把你在键盘敲的每个字符（包括回车）直接发过去，把收到的每个字节直接显示在屏幕上。

TCP Client code

复制代码

#include <QCoreApplication>
#include <QTcpSocket>
#include <QDebug>
int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QTcpSocket * socket = new QTcpSocket();

    QObject::connect(socket,&QTcpSocket::connected,[](){
        qDebug() << "connected";
    });
    QObject::connect(socket,&QTcpSocket::disconnected,[](){
        qDebug() << "disconnected";
        QCoreApplication::quit();
    });
    int num = 0;
    QObject::connect(socket,&QTcpSocket::readyRead,[&](){
        QByteArray data = socket->readAll();
        qDebug() << data;
        QString message = QString("Hello Server %1").arg(num);
        socket->write(message.toUtf8());
        num++;
    });
    socket->connectToHost("127.0.0.1",6000);

    if(socket->waitForConnected(5000))
    {
        QString message = "Hello Server";
        socket->write(message.toUtf8());
        socket->waitForBytesWritten(1000);
    }
    else
    {
        qDebug() << "连接超时" << socket->errorString();
        socket->deleteLater();
        return 1;
    }
    return a.exec();
}

功能

创建一个简单的TCP客户端

连接到指定IP和端口

发送"Hello Server!"消息

接收服务器响应打印并回复

流程

创建QTcpSocket对象

连接相关信号

调用connectToHost()连接到服务器

使用write()发送数据

在readyRead信号中读取数据

TCP Server 发送4MB文件

code

复制代码

#include <QCoreApplication>
#include <QTcpServer>
#include <QTcpSocket>
#include <QFile>
#include <chrono>
#include <thread>
#pragma pack (push,1)
struct FileInfo
{
    unsigned int allBytesNum;    // 总字节数  4M = 4 * 1024 * 1024
    unsigned int curPacketId;    // 当前包的编号
};
#pragma pack (pop)
int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QTcpServer server;
    if(!server.listen(QHostAddress::Any,6000))
    {
        qDebug() << __FILE__ << __LINE__ << "Server could not start!";
    }
    QObject::connect(&server,&QTcpServer::newConnection,[&](){
       QTcpSocket * clientSocket = server.nextPendingConnection();
       qDebug() << clientSocket->peerAddress().toString()
                << clientSocket->peerPort();

       QObject::connect(clientSocket,&QTcpSocket::readyRead,[clientSocket](){
           QByteArray data = clientSocket->readAll();
           qDebug() << "rec:" << data;

           QFile file("/data/LPL/Diff/Send/点阵.dat");
           //QFile file("/data/LPL/Diff/Send/main.cpp");
           bool ok = file.open(QIODevice::ReadOnly);
           qDebug() << __FILE__ << __LINE__ << ok;
           FileInfo fileInfo;
           fileInfo.allBytesNum = file.size();
           if(ok)
           {
               int num = 1;
               while(!file.atEnd())
               {
                   std::this_thread::sleep_for(std::chrono::milliseconds(2));
                   fileInfo.curPacketId = num;
                   QByteArray ba((const char *)&fileInfo,sizeof(fileInfo));
                   if(num % 100 == 1)
                   qDebug() << ba.toHex() << fileInfo.curPacketId;
                   ba.append(file.read(1024));
                   clientSocket->write(ba);
                   clientSocket->flush();
                   QCoreApplication::processEvents();
                   num++;
               }
               qDebug() << num - 1;
           }
           file.close();
       });
       QObject::connect(clientSocket,&QTcpSocket::disconnected,[clientSocket](){
           qDebug() << "Client disconnected: " << clientSocket->peerAddress().toString();
           clientSocket->deleteLater();
       });
    });
    return a.exec();
}

Tcp Client 接收4MB文件

复制代码

#include <QCoreApplication>
#include <QTcpSocket>
#include <QDebug>
#include <QFile>

#pragma pack (push,1)
struct FileInfo
{
    unsigned int allBytesNum;    // 总字节数  4M = 4 * 1024 * 1024
    unsigned int curPacketId;    // 当前包的编号
};
#pragma pack (pop)

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QTcpSocket * socket = new QTcpSocket();

    QObject::connect(socket,&QTcpSocket::connected,[](){
        qDebug() << "connected";
    });
    QObject::connect(socket,&QTcpSocket::disconnected,[](){
        qDebug() << "disconnected";
        QCoreApplication::quit();
    });
    QFile writedFile("/data/LPL/Diff/Rec/test8.dat");
    writedFile.open(QIODevice::WriteOnly);
    QObject::connect(socket,&QTcpSocket::readyRead,[&](){
        qDebug()<< " ready";
        while(socket->bytesAvailable() > 0)
        {
            QByteArray ba = socket->read(sizeof(FileInfo) + 1024);
            FileInfo * mFileInfo = (FileInfo *)ba.left(sizeof(FileInfo)).data();
            qDebug() << ba.left(sizeof(FileInfo)).toHex();
            int allPacketNum = mFileInfo->allBytesNum / 1024;
            if(mFileInfo->allBytesNum % 1024 > 0)
            {
                allPacketNum++;
            }
            writedFile.write(ba.remove(0,sizeof(FileInfo)));

            if(mFileInfo->curPacketId == allPacketNum)
            {
                qDebug() << "over";
                writedFile.close();
            }
        }
    });
    socket->connectToHost("127.0.0.1",6000);

    if(socket->waitForConnected(5000))
    {
        QString message = "Hello I am Client";
        socket->write(message.toUtf8());
        socket->waitForBytesWritten(1000);
    }
    else
    {
        qDebug() << "连接超时" << socket->errorString();
        socket->deleteLater();
        return 1;
    }
    return a.exec();
}

需要注意的地方：

使用QTcpSocket::readyRead时：

复制代码

        while(socket->bytesAvailable() > 0)
        {
		
		}

在发送的时候

复制代码

clientSocket->flush();
QCoreApplication::processEvents();

之前的现象：
发送端调用了4096次write，接收端没有触发readyRead信号。

原因：

write只是写入了缓冲区，没有实际发送

错误示例：

for(int i = 0;i < 4096;i++)

{

socket->write(data); // 只是写入缓冲区，没有实际发送

}

可以写成：

for(int i = 0;i < 4096;i++)

{

socket->write(data);

socket->flush(); //立刻刷新缓冲区

// QCoreApplication::processEvents();

// 之前还调用了processEvents(),但我删去后也没有影响。

}

qint64 QIODevice::write(const char *data, qint64 maxSize)

这个方法只是写入缓冲区吗？

是的，不是立刻发送。

实际发送时机：

1.Qt事件循环处理时 ------ 已测试可行

2.调用flush()时 ------ 已测试可行

3.缓冲区满时 ------ 可以理解

4.等待写入函数返回时 --- 不太理解

socket.flush();

立刻将缓冲区数据推送到操作系统网络栈

flush不保证对方已收到，只保证数据交给了操作系统。

for(int i = 0;i < 4096;i++)

{

socket->write(data);

QCoreApplication::processEvents();

// 处理事件循环，让Qt处理待发送的数据。

// 可行

}

// 等待字节写入

socket.waitForBytesWritten(1000);

这个函数需要了解一下

复制代码

[override virtual] bool QAbstractSocket::waitForBytesWritten(int msecs = 30000)
Reimplemented from QIODevice::waitForBytesWritten().
This function blocks until at least one byte has been written on the socket and the bytesWritten() signal has been emitted. The function will timeout after msecs milliseconds; the default timeout is 30000 milliseconds.
The function returns true if the bytesWritten() signal is emitted; otherwise it returns false (if an error occurred or the operation timed out).
Note: This function may fail randomly on Windows. Consider using the event loop and the bytesWritten() signal if your software will run on Windows.
See also waitForReadyRead().

阻塞等待msecs

如果所有缓冲数据被操作系统接受 ---> true

超时或发生错误 ---> false

我实践了一下：

复制代码

               while(!file.atEnd())
               {
                   //std::this_thread::sleep_for(std::chrono::milliseconds(2));
                   fileInfo.curPacketId = num;
                   QByteArray ba((const char *)&fileInfo,sizeof(fileInfo));
                   if(num % 100 == 1)
                   qDebug() << ba.toHex() << fileInfo.curPacketId;
                   ba.append(file.read(1024));
                   clientSocket->write(ba);
                   //clientSocket->flush();
                   //QCoreApplication::processEvents();
                   clientSocket->waitForBytesWritten(1);
                   num++;
               }

不使用flush

不使用processEvents

把延迟去掉
效果很好，速度很快,这个好用！！！

???

实际上是有问题的，数据不对了！

又测试了一次，确实有问题!

"10004000960d0000"

"10004000970d0000"

"10004000980d0000"

"10004000990d0000"

"100040009a0d0000"

ready

"64000000e023a0f7"

"64000000e0b370fb"

0d9a ---> 3482

后面的数据就乱掉了

是否是因为clientSocket->waitForBytesWritten(1);

1ms太短了

clientSocket->waitForBytesWritten(5);

改成5ms后，正常了！

感觉是比延时要快的，当然，这都是LocalHost

3ms，看起来也可以

同样的代码，使用两台机器测试，结果有问题！

延时 + flush ---> 依然有问题

经过测试，发现Client端之前的代码存在粘包问题。

我期望读到固定的头:

100040000010000000040000

但实际上读到的数据不是我想要的

原因：

TCP是流式协议

TCP传输的是字节流，没有固定的"消息边界"。

发送端多次write的数据，接收端可能一次全部收到、分多次收到或者与后续数据粘连。

解决方案：

使用固定长度数据头

复制代码

struct FileInfo
{
    unsigned int allBytesNum;    // 总字节数  4M = 4 * 1024 * 1024
    unsigned int curPacketId;    // 当前包的编号
    unsigned int curPacketSize;  // 当前包的大小
};

此时接收端可以先读取数据头，获取这个数据包的长度。

具体代码如下：

Server

复制代码

#include <QCoreApplication>
#include <QTcpServer>
#include <QTcpSocket>
#include <QFile>
#include <chrono>
#include <thread>
#pragma pack (push,1)
struct FileInfo
{
    unsigned int allBytesNum;    // 总字节数  4M = 4 * 1024 * 1024
    unsigned int curPacketId;    // 当前包的编号
    unsigned int curPacketSize;  // 当前包的大小
};
#pragma pack (pop)

#include <iostream>
int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);

    QTcpServer server;
    if(!server.listen(QHostAddress("192.9.2.100"),6000))
    {
        qDebug() << __FILE__ << __LINE__ << "Server could not start!";
    }
    QObject::connect(&server,&QTcpServer::newConnection,[&](){
       QTcpSocket * clientSocket = server.nextPendingConnection();
       qDebug() << clientSocket->peerAddress().toString()
                << clientSocket->peerPort();

       QObject::connect(clientSocket,&QTcpSocket::readyRead,[clientSocket](){
           QByteArray data = clientSocket->readAll();
           qDebug() << "rec:" << data;

           QFile file("/data/LPL/Diff/Send/点阵.dat");
           bool ok = file.open(QIODevice::ReadOnly);
           int allBytesNum = file.size();
           int remainderSize = allBytesNum;
           FileInfo fileInfo;
           fileInfo.allBytesNum = allBytesNum;
           if(ok)
           {
               int num = 1;
               while(!file.atEnd())
               {
                   std::this_thread::sleep_for(std::chrono::milliseconds(10));
                   //std::cin.get();
                   fileInfo.curPacketId = num;
                   if(remainderSize > 1024)
                   {
                       fileInfo.curPacketSize = 1024;
                   }
                   else
                   {
                        fileInfo.curPacketSize = remainderSize;
                   }
                   remainderSize -= fileInfo.curPacketSize;

                   QByteArray ba((const char *)&fileInfo,sizeof(fileInfo));
                   //if(num % 100 == 1)
                   qDebug() << ba.toHex() << fileInfo.curPacketId << fileInfo.curPacketSize;

                   ba.append(file.read(fileInfo.curPacketSize));
                   clientSocket->write(ba);
                   clientSocket->flush();
                   QCoreApplication::processEvents();
                   num++;
               }
               qDebug() << num - 1;
               qDebug() << "allBytesNum: "<< fileInfo.allBytesNum << file.size();
           }
           file.close();
       });
       QObject::connect(clientSocket,&QTcpSocket::disconnected,[clientSocket](){
           qDebug() << "Client disconnected: " << clientSocket->peerAddress().toString();
           clientSocket->deleteLater();
       });
    });
    return a.exec();
}

Client

复制代码

#include <QCoreApplication>
#include <QTcpSocket>
#include <QDebug>
#include <QFile>
#include <QDataStream>
#pragma pack (push,1)
struct FileInfo
{
    unsigned int allBytesNum;    // 总字节数  4M = 4 * 1024 * 1024
    unsigned int curPacketId;    // 当前包的编号
    unsigned int curPacketSize;  // 当前包的大小
};
#pragma pack (pop)

int main(int argc, char *argv[])
{
    QCoreApplication a(argc, argv);
    QTcpSocket * socket = new QTcpSocket();

    QObject::connect(socket,&QTcpSocket::connected,[](){
        qDebug() << "connected";
    });
    QObject::connect(socket,&QTcpSocket::disconnected,[](){
        qDebug() << "disconnected";
        QCoreApplication::quit();
    });
    int allPacketNum;
    int allBytesNum;
    int remainder;
    FileInfo nextBlockFileInfo;
    quint32 nextBlockSize = 0;
    bool isInit = false;
    QFile writedFile("/data/LPL/Diff/Rec/test1234.dat");
    //QFile writedFile("/data/LPL/Diff/Rec/main.cpp");
    writedFile.open(QIODevice::WriteOnly);
    QObject::connect(socket,&QTcpSocket::readyRead,[&](){
        if(nextBlockSize == 0)
        {
            if(socket->bytesAvailable() < sizeof(FileInfo))
            {
                // 长度信息还没收全
                return ;
            }
            QByteArray Head = socket->read(sizeof(FileInfo));
            nextBlockFileInfo = *(FileInfo *)Head.data();
            qDebug() <<Head.toHex() << nextBlockFileInfo.allBytesNum << nextBlockFileInfo.curPacketId << nextBlockFileInfo.curPacketSize;
            nextBlockSize = nextBlockFileInfo.curPacketSize;

            if(isInit == false)
            {
                isInit = true;
                allBytesNum = nextBlockFileInfo.allBytesNum;
                remainder = nextBlockFileInfo.allBytesNum;
                allPacketNum = nextBlockFileInfo.allBytesNum / 1024;
                if(nextBlockFileInfo.allBytesNum % 1024 > 0)
                {
                    allPacketNum++;
                }
            }
        }
        if(socket->bytesAvailable() < nextBlockSize)
        {
            // 数据包还没收全
            return ;
        }
        // 读取完整数据
        QByteArray ba  = socket->read(nextBlockFileInfo.curPacketSize);
        writedFile.write(ba);
        remainder -= nextBlockSize;
        nextBlockSize = 0;
        qDebug() << remainder;
        if(remainder == 0)
        {
            qDebug() << "allPacketNum:" << allPacketNum;
            qDebug() << "allBytesNum:" << allBytesNum;
            qDebug() << "over";
            writedFile.close();
        }
    });
    socket->connectToHost("192.9.2.100",6000);

    if(socket->waitForConnected(5000))
    {
        QString message = "Hello I am Client";
        socket->write(message.toUtf8());
        socket->waitForBytesWritten(1000);
    }
    else
    {
        qDebug() << "连接超时" << socket->errorString();
        socket->deleteLater();
        return 1;
    }
    return a.exec();
}

此时Client读取数据时，不会再出现粘包情况。

但有时候仍会出现丢包，一共4097个包，但有时候只能收到4090个左右。

且此时已经延时10ms * 4097 = 40.97s

这个4MB的文件，不应该传输这么慢的。

关于缓冲区

在网络通信中：

复制代码

// 发送方
socket->write(data); // 数据先进入发送缓冲区
->
// 操作系统网络栈
->
// 网络传输
->
// 接收方操作系统
->
// 接收方缓冲区 
->
// 接收方软件读取
socket->realAll(); // 从缓冲区读取

三层缓冲区模型

1.应用层缓冲区（Qt控制）

socket->setReadBufferSize(size);

2.操作系统Socket缓冲区

内核空间的缓冲区

virtual\] void QAbstractSocket::setSocketOption(QAbstractSocket::SocketOption option, const QVariant \&value) 更底层，操作系统管理 3.网络设备缓冲区 -网卡自己的缓冲区 -路由器/交换机的缓冲区 -用户程序无法控制 为什么需要缓冲区? 场景1：没有缓冲区 App:我要发数据! 操作系统: 网络卡顿，发不出去! 结果:程序卡住或数据丢失 场景2: 有缓冲区 App:我要发数据! 操作系统: 好，放缓冲区，我慢慢发 结果:程序继续运行，不卡顿 实际工作流程： 接收数据时： 网络 -\> 网卡 -\> 操作系统内核缓冲区 -\> Qt应用缓冲区 -\> App 发送数据时: 网络 \<- 网卡 \<- 操作系统内核缓冲区 \<- Qt应用缓冲区 \<- App ## 使用TCP进行4MB文件传输，有必要人工把它拆成一个个小包吗？ 不需要，也绝不应该人工拆包。TCP已经有完善的分包机制。 ## \[virtual\] void QAbstractSocket::setReadBufferSize(qint64 size) Sets the size of QAbstractSocket's internal read buffer to be size bytes. 内部的读取缓存 If the buffer size is limited to a certain size, QAbstractSocket won't buffer more than this size of data. Exceptionally, a buffer size of 0 means that the read buffer is unlimited and all incoming data is buffered. This is the default. This option is useful if you only read the data at certain points in time (e.g., in a real-time streaming application) or if you want to protect your socket against receiving too much data, which may eventually cause your application to run out of memory. Only QTcpSocket uses QAbstractSocket's internal buffer; QUdpSocket does not use any buffering at all, but rather relies on the implicit buffering provided by the operating system. Because of this, calling this function on QUdpSocket has no effect. See also readBufferSize() and read(). ## \[virtual\] QVariant QAbstractSocket::socketOption(QAbstractSocket::SocketOption option) Returns the value of the option option. This function was introduced in Qt 4.6. ## enum QAbstractSocket::SocketOption This enum represents the options that can be set on a socket. **If desired, they can be set after having received the connected() signal from the socket or after having received a new socket from a QTcpServer.** desired : 期望的 Constant Value Description QAbstractSocket::LowDelayOption 0 Try to optimize the socket for low latency. For a QTcpSocket this would set the TCP_NODELAY option and disable Nagle's algorithm. Set this to 1 to enable. latency : 延迟时间 no delay : 无延迟 NODELAY QAbstractSocket::KeepAliveOption 1 Set this to 1 to enable the SO_KEEPALIVE socket option alive : 活跃的 QAbstractSocket::MulticastTtlOption 2 Set this to an integer value to set IP_MULTICAST_TTL (TTL for multicast datagrams) socket option. QAbstractSocket::MulticastLoopbackOption 3 Set this to 1 to enable the IP_MULTICAST_LOOP (multicast loopback) socket option. QAbstractSocket::TypeOfServiceOption 4 This option is not supported on Windows. This maps to the IP_TOS socket option. For possible values, see table below. QAbstractSocket::SendBufferSizeSocketOption 5 Sets the socket send buffer size in bytes at the OS level. This maps to the SO_SNDBUF socket option. This option does not affect the QIODevice or QAbstractSocket buffers. This enum value has been introduced in Qt 5.3. QAbstractSocket::ReceiveBufferSizeSocketOption 6 Sets the socket receive buffer size in bytes at the OS level. This maps to the SO_RCVBUF socket option. This option does not affect the QIODevice or QAbstractSocket buffers (see setReadBufferSize()). This enum value has been introduced in Qt 5.3. QAbstractSocket::PathMtuSocketOption 7 Retrieves the Path Maximum Transmission Unit (PMTU) value currently known by the IP stack, if any. Some IP stacks also allow setting the MTU for transmission. This enum value was introduced in Qt 5.11.