Note:文章待完结
文章目录
-
- 前言
- 一、整体目录结构
- 二、driver包
-
- 1、驱动相关driver.Driver
- 2、驱动连接:driver.Conn
- 3、预处理结构:Stmt
- [4、执行结果 driver.Result](#4、执行结果 driver.Result)
- 5、查询结果:driver.Rows
- 6、driver.RowsAffected
- 7、driver.Value
- 8、Value定义转换相关
- 三、sql包
- 二、结语
- 三、参考
前言
在golang中,我们比较熟悉的mysql相关的库就是database/sql,这是golang的内置库,该标准库没有具体实现,只列出第三方库需要实现的具体内容。也就是说,这个库只是定义了接口,并没有具体的实现。Go语言为开发数据库驱动定义了一些标准接口,使用标准接口开发的代码,在迁移数据库时,不需要做任何修改(当然双方数据库都遵守标准接口)。下面我将基于golang1.19的源码探究这个库的实现。
源码地址:https://github.com/golang/go/tree/release-branch.go1.19/src/database/sql
一、整体目录结构
整个目录结构就是这样,包含两个包:sql和driver,这两个包必须一起配合着使用,sql包中主要包含着数据库具体实例、驱动的注册、结果集读取、转换各种定义类型结构等。driver包中主要是与数据库打交道的部分,增删改查的接口定义就在这里面。
sql包:
driver包:
二、driver包
在driver包中,主要有如下的接口定义:
Connector
:抽象的数据库连接器,需要具备创建数据库连接以及返回从属的数据库驱动的能力。Driver
:抽象的数据库驱动,具备创建数据库连接的能力。Conn
:抽象的数据库连接,具备预处理 sql 以及开启事务的能力。Tx
:抽象的事务,具备提交和回滚的能力。Statement
:抽象的请求预处理状态. 具备实际执行 sql 并返回执行结果的能力。Result/Row
:抽象的 sql 执行结果。
1、驱动相关driver.Driver
Driver是一个数据库驱动的接口,定义了 Open(name string) ,该方法返回一个数据库的Conn接口:
go
// Driver is the interface that must be implemented by a database
// driver.
//
// Database drivers may implement DriverContext for access
// to contexts and to parse the name only once for a pool of connections,
// instead of once per connection.
type Driver interface {
// Open returns a new connection to the database.
// The name is a string in a driver-specific format.
//
// Open may return a cached connection (one previously
// closed), but doing so is unnecessary; the sql package
// maintains a pool of idle connections for efficient re-use.
//
// The returned connection is only used by one goroutine at a
// time.
Open(name string) (Conn, error)
}
在上面的源码中,我们可以清晰知道,Driver接口是必须要被所有的数据库驱动程序实现的,提供而一个Open方法用于返回一个连接,这个连接可能是缓存的有效的,也可能是新建的连接。同时也提供了一个DriverContext接口,数据库驱动程序可以实现DriverContext以访问上下文,并仅为连接池解析一次名称,而不是每个连接解析一次。
DriverContext接口提供了一个OpenConnector方法用于返回一个连接器,在连接器中去获取对应的连接。连接器接口Connector提供了两个方法,Connect和Driver,其中Connect用于获取连接,并且可以附带参数ctx,Driver用于获取当前这个连接器的的驱动程序。
go
// If a Driver implements DriverContext, then sql.DB will call
// OpenConnector to obtain a Connector and then invoke
// that Connector's Connect method to obtain each needed connection,
// instead of invoking the Driver's Open method for each connection.
// The two-step sequence allows drivers to parse the name just once
// and also provides access to per-Conn contexts.
type DriverContext interface {
// OpenConnector must parse the name in the same format that Driver.Open
// parses the name parameter.
OpenConnector(name string) (Connector, error)
}
// A Connector represents a driver in a fixed configuration
// and can create any number of equivalent Conns for use
// by multiple goroutines.
//
// A Connector can be passed to sql.OpenDB, to allow drivers
// to implement their own sql.DB constructors, or returned by
// DriverContext's OpenConnector method, to allow drivers
// access to context and to avoid repeated parsing of driver
// configuration.
//
// If a Connector implements io.Closer, the sql package's DB.Close
// method will call Close and return error (if any).
type Connector interface {
// Connect returns a connection to the database.
// Connect may return a cached connection (one previously
// closed), but doing so is unnecessary; the sql package
// maintains a pool of idle connections for efficient re-use.
//
// The provided context.Context is for dialing purposes only
// (see net.DialContext) and should not be stored or used for
// other purposes. A default timeout should still be used
// when dialing as a connection pool may call Connect
// asynchronously to any query.
//
// The returned connection is only used by one goroutine at a
// time.
Connect(context.Context) (Conn, error)
// Driver returns the underlying Driver of the Connector,
// mainly to maintain compatibility with the Driver method
// on sql.DB.
Driver() Driver
}
接下来就是驱动的注册了,database/sql提供了一个驱动注册静态方法,驱动的具体实现中,可以调用该方法注册相关的驱动,但同时只允许注册同一类型的驱动,否则会panic。
go
func Register(name string, driver driver.Driver) {
driversMu.Lock()
defer driversMu.Unlock()
if driver == nil {
panic("sql: Register driver is nil")
}
if _, dup := drivers[name]; dup {
panic("sql: Register called twice for driver " + name)
}
drivers[name] = driver
}
该函数用来注册数据库驱动。当第三方开发者开发数据库驱动时,都会实现init函数,而init中会调用 Register(name string, driver driver.Driver) 完成驱动注册。如<github.com/go-sql-driver/mysql>
2、驱动连接:driver.Conn
go
type Conn interface {
// Prepare returns a prepared statement, bound to this connection.
Prepare(query string) (Stmt, error)
// Close invalidates and potentially stops any current
// prepared statements and transactions, marking this
// connection as no longer in use.
//
// Because the sql package maintains a free pool of
// connections and only calls Close when there's a surplus of
// idle connections, it shouldn't be necessary for drivers to
// do their own connection caching.
//
// Drivers must ensure all network calls made by Close
// do not block indefinitely (e.g. apply a timeout).
Close() error
// Begin starts and returns a new transaction.
//
// Deprecated: Drivers should implement ConnBeginTx instead (or additionally).
Begin() (Tx, error)
}
Prepare
:返回与当前连接相关的执行SQL语句的准备状态(Stmt),可以进行查询、删除等操作。
Close
:关闭当前的链接,执行释放连接拥有的资源等清理工作。
Begin
: // 返回一个代表事务处理的Tx,通过它可以进行查询、更新等操作,或者对事务进行回滚、递交。
新版本中,Begin方法已经不推荐了,被ConnBeginTx代替了, 新版本中的Begin方法多提供了入参ctx和额外的可选参数opts,便于扩展和控制。
go
// ConnBeginTx enhances the Conn interface with context and TxOptions.
type ConnBeginTx interface {
// BeginTx starts and returns a new transaction.
// If the context is canceled by the user the sql package will
// call Tx.Rollback before discarding and closing the connection.
//
// This must check opts.Isolation to determine if there is a set
// isolation level. If the driver does not support a non-default
// level and one is set or if there is a non-default isolation level
// that is not supported, an error must be returned.
//
// This must also check opts.ReadOnly to determine if the read-only
// value is true to either set the read-only transaction property if supported
// or return an error if it is not supported.
BeginTx(ctx context.Context, opts TxOptions) (Tx, error)
}
3、预处理结构:Stmt
go
// Stmt is a prepared statement. It is bound to a Conn and not
// used by multiple goroutines concurrently.
type Stmt interface {
// Close closes the statement.
//
// As of Go 1.1, a Stmt will not be closed if it's in use
// by any queries.
//
// Drivers must ensure all network calls made by Close
// do not block indefinitely (e.g. apply a timeout).
Close() error
// NumInput returns the number of placeholder parameters.
//
// If NumInput returns >= 0, the sql package will sanity check
// argument counts from callers and return errors to the caller
// before the statement's Exec or Query methods are called.
//
// NumInput may also return -1, if the driver doesn't know
// its number of placeholders. In that case, the sql package
// will not sanity check Exec or Query argument counts.
NumInput() int
// Exec executes a query that doesn't return rows, such
// as an INSERT or UPDATE.
//
// Deprecated: Drivers should implement StmtExecContext instead (or additionally).
Exec(args []Value) (Result, error)
// Query executes a query that may return rows, such as a
// SELECT.
//
// Deprecated: Drivers should implement StmtQueryContext instead (or additionally).
Query(args []Value) (Rows, error)
}
// StmtExecContext enhances the Stmt interface by providing Exec with context.
type StmtExecContext interface {
// ExecContext executes a query that doesn't return rows, such
// as an INSERT or UPDATE.
//
// ExecContext must honor the context timeout and return when it is canceled.
ExecContext(ctx context.Context, args []NamedValue) (Result, error)
}
// StmtQueryContext enhances the Stmt interface by providing Query with context.
type StmtQueryContext interface {
// QueryContext executes a query that may return rows, such as a
// SELECT.
//
// QueryContext must honor the context timeout and return when it is canceled.
QueryContext(ctx context.Context, args []NamedValue) (Rows, error)
}
Close
:关闭当前的连接状态,但如果当前正在执行query,query还是会有效返回rows数据。
NumInput
:返回当前预留参数的个数,当返回>=0时,数据库驱动会智能检查调用者的参数。 当数据库驱动包不知道预留参数的时候,返回-1。
Exec
:执行Prepare准备好的SQL,传入参数执行Update/Insert等操作,返回Result数据,Result中包含最后插入的自增主键序号(LastInsertId)和受影响的行数(RowAffected)。
Query
:执行Prepare准备好的SQL,传入需要的参数执行select操作,返回Rows结果集。
4、执行结果 driver.Result
go
// Result is the result of a query execution.
type Result interface {
// LastInsertId returns the database's auto-generated ID
// after, for example, an INSERT into a table with primary
// key.
LastInsertId() (int64, error)
// RowsAffected returns the number of rows affected by the
// query.
RowsAffected() (int64, error)
}
5、查询结果:driver.Rows
go
// Rows is an iterator over an executed query's results.
type Rows interface {
// 该函数返回查询数据库表的字段信息,这个返回的slice和SQL查询的字段一一对应,
// 而不是返回整张表的所有字段。
Columns() []string
// 用来关闭Rows迭代器
Close() error
// 该函数用来返回下一条数据,把数据赋值给dest .
// dest里面元素必须是driver.Value的值(string除外),返回的数据里面所有的 string 都必须转换成
// []byte.如果最后没有数据了,Next 函数返回 io.EOF。
Next(dest []Value) error
}
可以看到,在新版的源码中,Exec和Query已经被单独拎出去定义了接口,方法中只是为了增加ctx参数,这也是golang为了保持向下兼容而做的,试想,如果直接在原有的接口定义的加入ctx,升级golang版本的时候这块儿肯定得花很大功夫去改造。
6、driver.RowsAffected
RowsAffected 不是别的东西,实际上只是 int64 的别名,但它实现了Result接口,用于底层实现 Result 的表示方式,构建Exec方法返回的结果集。
go
// RowsAffected implements Result for an INSERT or UPDATE operation
// which mutates a number of rows.
type RowsAffected int64
var _ Result = RowsAffected(0)
func (RowsAffected) LastInsertId() (int64, error) {
return 0, errors.New("LastInsertId is not supported by this driver")
}
func (v RowsAffected) RowsAffected() (int64, error) {
return int64(v), nil
}
7、driver.Value
Value 其实是一个空接口,可以容纳任何的数据。
go
// diver 的 Value 是驱动必须能够操作的 Value,Value要么是nil,要么是下面任意一种:
//
// int64
// float64
// bool
// []byte
// string [*] 除了Rows.Next,返回的不能是string
// time.Time
//
type Value interface{}
8、Value定义转换相关
在driver/types.go中,还定义了ValueConverter将一个普通的值(any)转换成driver.Value的接口、Valuer接口用于获取driver.Value等,就不逐个展开了。
go
// ValueConverter is the interface providing the ConvertValue method.
//
// Various implementations of ValueConverter are provided by the
// driver package to provide consistent implementations of conversions
// between drivers. The ValueConverters have several uses:
//
// - converting from the Value types as provided by the sql package
// into a database table's specific column type and making sure it
// fits, such as making sure a particular int64 fits in a
// table's uint16 column.
//
// - converting a value as given from the database into one of the
// driver Value types.
//
// - by the sql package, for converting from a driver's Value type
// to a user's type in a scan.
type ValueConverter interface {
// ConvertValue converts a value to a driver Value.
ConvertValue(v any) (Value, error)
}
// Valuer is the interface providing the Value method.
//
// Types implementing Valuer interface are able to convert
// themselves to a driver Value.
type Valuer interface {
// Value returns a driver Value.
// Value must not panic.
Value() (Value, error)
}
三、sql包
在sql包中,主要是sql.go中的 DB结构,对应为数据库的具象化实例。DB
go
// DB is a database handle representing a pool of zero or more
// underlying connections. It's safe for concurrent use by multiple
// goroutines.
//
// The sql package creates and frees connections automatically; it
// also maintains a free pool of idle connections. If the database has
// a concept of per-connection state, such state can be reliably observed
// within a transaction (Tx) or connection (Conn). Once DB.Begin is called, the
// returned Tx is bound to a single connection. Once Commit or
// Rollback is called on the transaction, that transaction's
// connection is returned to DB's idle connection pool. The pool size
// can be controlled with SetMaxIdleConns.
type DB struct {
// Atomic access only. At top of struct to prevent mis-alignment
// on 32-bit platforms. Of type time.Duration.
waitDuration int64 // Total time waited for new connections.
connector driver.Connector
// numClosed is an atomic counter which represents a total number of
// closed connections. Stmt.openStmt checks it before cleaning closed
// connections in Stmt.css.
numClosed uint64
mu sync.Mutex // protects following fields
freeConn []*driverConn // free connections ordered by returnedAt oldest to newest
connRequests map[uint64]chan connRequest
nextRequest uint64 // Next key to use in connRequests.
numOpen int // number of opened and pending open connections
// Used to signal the need for new connections
// a goroutine running connectionOpener() reads on this chan and
// maybeOpenNewConnections sends on the chan (one send per needed connection)
// It is closed during db.Close(). The close tells the connectionOpener
// goroutine to exit.
openerCh chan struct{}
closed bool
dep map[finalCloser]depSet
lastPut map[*driverConn]string // stacktrace of last conn's put; debug only
maxIdleCount int // zero means defaultMaxIdleConns; negative means 0
maxOpen int // <= 0 means unlimited
maxLifetime time.Duration // maximum amount of time a connection may be reused
maxIdleTime time.Duration // maximum amount of time a connection may be idle before being closed
cleanerCh chan struct{}
waitCount int64 // Total number of connections waited for.
maxIdleClosed int64 // Total number of connections closed due to idle count.
maxIdleTimeClosed int64 // Total number of connections closed due to idle time.
maxLifetimeClosed int64 // Total number of connections closed due to max connection lifetime limit.
stop func() // stop cancels the connection opener.
}
几个主要的字段,其他字段大部分都是和连接池参数相关的:
connector
:用于创建数据库连接的抽象连接器,由第三方数据库提供具体实现。
mu
:互斥锁,保证并发安全。
freeConn
:数据库连接池,缓存可用的连接以供后续复用。
connRequests
:唤醒通道集合,和阻塞等待连接的协程是一对一的关系。
openerCh
:创建连接信号通道. 用于向连接创建协程 opener goroutine 发送信号。
stop
:连接创建协程 opener goroutine 的终止器,用于停止该协程。
DB结构主要作用如下:
二、结语
本章中我们学习了,golang中database/sql的源码阅读,了解了整个database/sql最大的特点就是定义接口,不做具体实现,从而让使用方去方便使用不同的驱动实现。同时提供了DB实例,内置连接池,方便管理连接的创建和销毁。