PostgreSQL源码分析——initdb

数据库初始化

在安装完数据库后,需要进行初始化数据库操作,对应PostgreSQL数据库中就是需要进行initdb后,才能对数据库进行启动。initdb的过程,其实就是创建数据库实例的过程,生成模板数据库和相应的目录、文件信息,系统表也是在这个阶段生成的。我们想一下,数据库运行都需要什么? 数据库是用来存取数据的,需要有存放数据的目录,执行过程中需要存放WAL日志,需要pg_wal目录存放日志,需要配置文件,最重要的是数据库怎么把一些文件抽象为表,表在数据库中存储形式就是文件,那么怎么对文件进行"解码"呢?那就是系统表,通过系统表,我们知道表中有多少列,每个列是什么类型,有什么约束等等。可以说,系统表是至关重要的。而initdb最重要的事情之一,就是生成系统表。

我们先看一下initdb后,

c++ 复制代码
postgres@slpc:~/pgsql$ ./bin/initdb -D pgdata/
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.

The database cluster will be initialized with locales
  COLLATE:  en_US.UTF-8
  CTYPE:    en_US.UTF-8
  MESSAGES: en_US.UTF-8
  MONETARY: zh_CN.UTF-8
  NUMERIC:  zh_CN.UTF-8
  TIME:     zh_CN.UTF-8
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".

Data page checksums are disabled.

fixing permissions on existing directory pgdata ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Asia/Shanghai
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok

initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.

Success. You can now start the database server using:

    ./bin/pg_ctl -D pgdata/ -l logfile start

postgres@slpc:~/pgsql$ cd pgdata/ && ls
base           pg_logical    pg_stat      pg_wal
global         pg_multixact  pg_stat_tmp  pg_xact
pg_commit_ts   pg_notify     pg_subtrans  postgresql.auto.conf
pg_dynshmem    pg_replslot   pg_tblspc    postgresql.conf
pg_hba.conf    pg_serial     pg_twophase
pg_ident.conf  pg_snapshots  PG_VERSION
源码分析

PostgreSQL初始化流程:

  1. 编译阶段,由genbki.pl脚本读取系统表定义文件(src/backend/catalog/pg_*.h),生成postgres.bki文件
perl 复制代码
# genbki.pl
#    Perl script that generates postgres.bki and symbol definition
#    headers from specially formatted header files and data files.
#    postgres.bki is used to initialize the postgres template database.
  1. initdb创建目录
  2. initdb生成template1数据库
  3. 由template1生成template0和postgres数据库

其核心的说明可以参考其代码注释:

c++ 复制代码
/* initdb --- initialize a PostgreSQL installation
 *
 * initdb creates (initializes) a PostgreSQL database cluster (site,
 * instance, installation, whatever).  A database cluster is a
 * collection of PostgreSQL databases all managed by the same server.
 *
 * To create the database cluster, we create the directory that contains
 * all its data, create the files that hold the global tables, create
 * a few other control files for it, and create three databases: the
 * template databases "template0" and "template1", and a default user
 * database "postgres".
 *
 * The template databases are ordinary PostgreSQL databases.  template0
 * is never supposed to change after initdb, whereas template1 can be
 * changed to add site-local standard data.  Either one can be copied
 * to produce a new database.
 *
 * For largely-historical reasons, the template1 database is the one built
 * by the basic bootstrap process.  After it is complete, template0 and
 * the default database, postgres, are made just by copying template1.
 *
 * To create template1, we run the postgres (backend) program in bootstrap
 * mode and feed it data from the postgres.bki library file.  After this
 * initial bootstrap phase, some additional stuff is created by normal
 * SQL commands fed to a standalone backend.  Some of those commands are
 * just embedded into this program (yeah, it's ugly), but larger chunks
 * are taken from script files.

现在我们分析一下initdb的源码,核心代码在src/bin/initdb/initdb.c中, 源码的解析可以参考《PostgreSQL数据库内核分析》第2章。

c++ 复制代码
main(int argc, char *argv[])
--> atexit(cleanup_directories_atexit);    // 如果执行失败,清除已创建的目录文件
--> setup_pgdata();                     // 获取PGDATA目录,或从-D 中获取
--> setup_data_file_paths();        
--> initialize_data_directory();
    --> create_data_directory();        // 创建PGDATA目录
    --> create_xlog_or_symlink();       // 创建pg_wal目录
    --> mkdir(path, pg_dir_create_mode) //创建其他的目录,base,global, pg_xact等等
    --> write_version_file(NULL);       // 创建PG_VERSION文件,写入主版本号, 数据库启动时会检查应用程序与实例版本好是否兼容
    --> set_null_conf();                // 创建配置文件postgresql.conf
	--> test_config_settings();
	--> setup_config();                 // 写配置文件,postgresql.conf pg_hba.conf postgresql.auto.conf
    --> bootstrap_template1();          // run the BKI script in bootstrap mode to create template1,数据存储在base/1中
        --> snprintf(cmd, sizeof(cmd), "\"%s\" --boot -x1 -X %u %s %s %s %s", backend_exec, wal_segment_size_mb * (1024 * 1024), data_checksums ? "-k" : "", boot_options, extra_options, debug ? "-d 5" : "");
        --> PG_CMD_OPEN;
        	for (line = bki_lines; *line != NULL; line++)
	        {
		        PG_CMD_PUTS(*line);
		        free(*line);
	        }
        --> PG_CMD_CLOSE;

    --> write_version_file("base/1");
    --> setup_auth(cmdfd);
	--> setup_run_file(cmdfd, system_constraints_file);
	--> setup_run_file(cmdfd, system_functions_file);
	--> setup_depend(cmdfd);
    --> setup_run_file(cmdfd, system_views_file);
    --> load_plpgsql(cmdfd);            // load PL/pgSQL server-side language
	--> vacuum_db(cmdfd);
	--> make_template0(cmdfd);
    -->	make_postgres(cmdfd);

创建数据库其实并不是initdb独立去完成的,只是initdb向Postgres进程发送命令,由postgres进程通过--boot进入特殊的bootstrap模式运行执行。bootstrap_template1,这个比较重要,主要是启动postgres进程,执行postgres.bki中特殊的语句,创建系统表。这里说明一下,PG_CMD_OPENPG_CMD_PUTSPG_CMD_CLOSE是什么意思:

c++ 复制代码
#define PG_CMD_OPEN \
do { \
	cmdfd = popen_check(cmd, "w"); \
	if (cmdfd == NULL) \
		exit(1); /* message already printed by popen_check */ \
} while (0)

/* Open a subcommand with suitable error messaging */
static FILE * popen_check(const char *command, const char *mode)
{
	FILE	   *cmdfd;

	fflush(stdout);
	fflush(stderr);
	errno = 0;
	cmdfd = popen(command, mode);
	if (cmdfd == NULL)
		pg_log_error("could not execute command \"%s\": %m", command);
	return cmdfd;
}

实质是调用popen函数去执行指定的命令。

最后,借用阿里云的分享文档,已加深大家对此的理解。详细文档见PostgreSQL体系结构

参考文档:

《PostgreSQL数据库内核分析》 第2章

《openGauss数据库源码解析》 第3章

相关推荐
零炻大礼包26 分钟前
【SQL server】数据库远程连接配置
数据库
zmgst35 分钟前
canal1.1.7使用canal-adapter进行mysql同步数据
java·数据库·mysql
随心............36 分钟前
python操作MySQL以及SQL综合案例
数据库·mysql
€☞扫地僧☜€37 分钟前
docker 拉取MySQL8.0镜像以及安装
运维·数据库·docker·容器
CopyDragon42 分钟前
设置域名跨越访问
数据库·sqlite
xjjeffery43 分钟前
MySQL 基础
数据库·mysql
写bug的小屁孩1 小时前
前后端交互接口(三)
运维·服务器·数据库·windows·用户界面·qt6.3
恒辉信达1 小时前
hhdb数据库介绍(8-4)
服务器·数据库·mysql
齐 飞2 小时前
MongoDB笔记01-概念与安装
前端·数据库·笔记·后端·mongodb
云空2 小时前
《Python 与 SQLite:强大的数据库组合》
数据库·python·sqlite