build-your-own-x学习笔记——Updating

项目地址：https://github.com/codecrafters-io/build-your-own-x

（一）Write your own miniature Redis with Python

The other day the idea occurred to me that it would be neat to write a simple Redis-like database server. While I've had plenty of experience with WSGI applications, a database server presented a novel challenge and proved to be a nice practical way of learning how to work with sockets in Python. In this post I'll share what I learned along the way.

The goal of my project was to write a simple server that I could use with a task queue project of mine called huey. Huey uses Redis as the default storage engine for tracking enqueued jobs, results of finished jobs, and other things. For the purposes of this post, I've reduced the scope of the original project even further so as not to muddy the waters with code you could very easily write yourself, but if you're curious, you can check out the end result here (documentation).

The server we'll be building will be able to respond to the following commands:

GET <key>

SET <key> <value>

DELETE <key>

FLUSH

MGET <key1> ... <keyn>

MSET <key1> <value1> ... <keyn> <valuen>

We'll support the following data-types as well:

Strings and Binary Data

Numbers

NULL

Arrays (which may be nested)

Dictionaries (which may be nested)

Error messages

To handle multiple clients asynchronously, we'll be using gevent, but you could also use the standard library's SocketServer module with either the ForkingMixin or the ThreadingMixin.

大意：作者认为实现一个小型的Redis是一件新颖并且具有实践意义的工作，是一个通过Sockets 学习网络编程 的绝佳实践。项目的任务是写一个简单的服务端可以追踪入队的任务、完成工作的结果和其他事物。

服务端可以响应以下任务：

GET <key>
SET <key> <value>
DELETE <key>
FLUSH
MGET <key1> ... <keyn>
MSET <key1> <value1> ... <keyn> <valuen>

支持以下数据类型：

Strings and Binary Data
Numbers
NULL
Arrays (which may be nested)
Dictionaries (which may be nested)
Error messages

为了异步处理多客户端，将使用gevent(基于协程的python网络库，协程：一个可以随时暂停去处理别的事物，等会儿还能原地复活继续干活的超级函数)

Skeleton

Let's frame up a skeleton for our server. We'll need the server itself, and a callback to be executed when a new client connects. Additionally we'll need some kind of logic to process the client request and to send a response.

Here's a start:
复制代码
from gevent import socket
from gevent.pool import Pool
from gevent.server import StreamServer

from collections import namedtuple
from io import BytesIO
from socket import error as socket_error


# We'll use exceptions to notify the connection-handling loop of problems.
class CommandError(Exception): pass
class Disconnect(Exception): pass

Error = namedtuple('Error', ('message',))


class ProtocolHandler(object):
    def handle_request(self, socket_file):
        # Parse a request from the client into it's component parts.
        pass

    def write_response(self, socket_file, data):
        # Serialize the response data and send it to the client.
        pass


class Server(object):
    def __init__(self, host='127.0.0.1', port=31337, max_clients=64):
        self._pool = Pool(max_clients)
        self._server = StreamServer(
            (host, port),
            self.connection_handler,
            spawn=self._pool)

        self._protocol = ProtocolHandler()
        self._kv = {}

    def connection_handler(self, conn, address):
        # Convert "conn" (a socket object) into a file-like object.
        socket_file = conn.makefile('rwb')

        # Process client requests until client disconnects.
        while True:
            try:
                data = self._protocol.handle_request(socket_file)
            except Disconnect:
                break

            try:
                resp = self.get_response(data)
            except CommandError as exc:
                resp = Error(exc.args[0])

            self._protocol.write_response(socket_file, resp)

    def get_response(self, data):
        # Here we'll actually unpack the data sent by the client, execute the
        # command they specified, and pass back the return value.
        pass

    def run(self):
        self._server.serve_forever()
The above code is hopefully fairly clear. We've separated concerns so that the protocol handling is in it's own class with two public methods: handle_request and write_response. The server itself uses the protocol handler to unpack client requests and serialize server responses back to the client. The get_response() method will be used to execute the command initiatied by the client.

Taking a closer look at the code of the connection_handler() method, you can see that we obtain a file-like wrapper around the socket object. This wrapper allows us to abstract away some of the quirks one typically encounters working with raw sockets. The function enters an endless loop, reading requests from the client, sending responses, and finally exiting the loop when the client disconnects (indicated by read() returning an empty string).

We use typed exceptions to handle client disconnects and to notify the user of errors processing commands. For example, if the user makes an improperly formatted request to the server, we will raise a CommandError, which is serialized into an error response and sent to the client.

Before going further, let's discuss how the client and server will communicate.