测试崖山-DM8-GaussDB-OpenTeleDB数据库在长事务下表和索引是否膨胀

robinson19882026-01-09 18:46

先给大家分享下本人最近几年遇到的PG系数据库表和索引膨胀CASE吧

1.原生PG，某制造业mes系统高频create/drop table，引起pg_class等系统表膨胀，系统运行2到3个月之后，查询数据字典的SQL从跑几十ms下降到几百ms甚至到秒级。和开发沟通，使用临时表替换高频create/drop table，但是开发说开发和测试工作量太大，不改。最终被迫每2-3个月停业务对数据字典进行vacuum full操作

2.openGauss系数据库某发行版，一制造业客户遇到了索引膨胀问题，客户每个月月初要做月结，每次月结都会卡死2小时，客户一直在抱怨后面定位到是索引膨胀，索引上死元组有3到4000W行，索引是十几个字段组成的联合主键，由于死元组有几千万行，批量INSERT到该表，每插入一行数据就要进行唯一性检查，死元组太多了，外加上联合主键的原因导致唯一性检查特别慢，就这一步卡死2小时，正常情况下几十秒跑完。最开始尝试使用vacuum full回收空间，但是由于有长事务，vacuum full回收不了空间，pg_repack也回收不了，最终被逼折腾出人工回收空间的方法，搞个了crontab，每隔一段时间检查n_dead_tup，当n_dead_tup大于某个阈值，检查表上有没有锁，没有锁就人工回收空间，自此之后，每次月结跑批就很稳定了表膨胀人工处理可以参考本人这篇博文 https://blog.csdn.net/robinson1988/article/details/147063836?spm=1001.2014.3001.5501

3.openGauss系数据库某发行版，遇到个bug，autovacuum进程一直对某个表空转，其余表长时间没来得及做autovacuum，最终全库表膨胀，CPU一直100%

4.移动磐维遇到n起表膨胀导致SQL性能问题案例，最后也是通过表膨胀人工处理解决

5.openGauss系数据库某发行版，某交易所开始未作长事务管控，导致某个中频UPDATE功能越跑越慢，最后做了长事务控制得以缓解

...等等等等...

因为本人是专门搞性能优化的，出现了严重的表和索引膨胀一定会引起性能问题，出现了性能问题又多半会找我，所以本人对表和索引膨胀变得特别敏感了

好了废话不多说了，现在开始测试

数据库切换到YASHAN23.5.1

1.构造测试数据

复制代码 
create table t  as select * from test01 where 1=0;
create index idx_t_n1 on t(object_id);
create index idx_t_n2 on t(object_id,owner,object_name,created,status,data_object_id);
insert into t select * from test01 where rownum<=1000000;
commit;

2.查看表和索引大小

复制代码 
select owner, segment_name, bytes / 1024 / 1024
  from dba_segments
 where segment_name in ('T', 'IDX_T_N1', 'IDX_T_N2');


OWNER                          SEGMENT_NAME                   BYTES/1024/1024 
------------------------------ ------------------------------ --------------- 
SCOTT                          IDX_T_N2                                   192
SCOTT                          IDX_T_N1                                    20
SCOTT                          T                                          116

3.模拟长事务不提交

复制代码 
session1:

update t set owner='A' where object_id=2;

4.对表 delete,insert,update 30次

复制代码 
session2:

set serveroutput on
begin
  for i in 1..30 loop
    dbms_output.put_line(i || ': ' || sysdate);
    update t set owner='B' where object_id>2; 
    commit;
    delete from t where object_id>2;
    commit;
    insert into t select * from test01 where object_id>2 and rownum<=999988;
    commit;
  end loop;
end;
/

5.查看表大小

复制代码 
select owner, segment_name, bytes / 1024 / 1024
  from dba_segments
 where segment_name in ('T', 'IDX_T_N1', 'IDX_T_N2');
OWNER                          SEGMENT_NAME                   BYTES/1024/1024 
------------------------------ ------------------------------ --------------- 
SCOTT                          IDX_T_N2                                   368
SCOTT                          IDX_T_N1                                    20
SCOTT                          T                                          116

数据库切换到DM8 DB Version: 0x7000d

1.构造测试数据

复制代码 
create table t  as select * from test01 where 1=0;
create index idx_t_n1 on t(object_id);
create index idx_t_n2 on t(object_id,owner,object_name,created,status,data_object_id);
insert into t select * from test01 where rownum<=1000000;
commit;

2.查看表大小

复制代码 
SQL> select owner, segment_name, bytes / 1024 / 1024
  from dba_segments
 where segment_name in ('T', 'IDX_T_N1', 'IDX_T_N2');2   3   

LINEID     OWNER SEGMENT_NAME BYTES/1024/1024     
---------- ----- ------------ --------------------
1          SCOTT IDX_T_N1     42
2          SCOTT IDX_T_N2     123
3          SCOTT T            110

3.模拟长事务不提交

复制代码 
session1:

update t set owner='A' where object_id=2;

4.对表 delete,insert,update 30次

复制代码 
session2:

set serveroutput on
begin
  for i in 1..30 loop
    dbms_output.put_line(i || ': ' || sysdate);
    update t set owner='B' where object_id>2; 
    commit;
    delete from t where object_id>2;
    commit;
    insert into t select * from test01 where object_id>2 and rownum<=999988;
    commit;
  end loop;
end;
/

5.查看表大小

复制代码 
SQL> select owner, segment_name, bytes / 1024 / 1024
  from dba_segments
 where segment_name in ('T', 'IDX_T_N1', 'IDX_T_N2');2   3   

LINEID     OWNER SEGMENT_NAME BYTES/1024/1024     
---------- ----- ------------ --------------------
1          SCOTT IDX_T_N1     170
2          SCOTT IDX_T_N2     776
3          SCOTT T            304

数据库切换到GaussDB，建表的时候存储类型选择USTORE

复制代码 
gaussdb=# \d+
                                                                 List of relations
 Schema |   Name   | Type  |  Owner  |  Size   |                                       Storage                                       | Description 
--------+----------+-------+---------+---------+-------------------------------------------------------------------------------------+-------------
 public | t_ustore | table | gaussdb | 126 MB  | {orientation=row,storage_type=ustore,compression=no,segment=off,parallel_workers=8} | 
 public | test01   | table | gaussdb | 5614 MB | {orientation=row,compression=no,storage_type=USTORE,segment=off}                    | 
 public | test02   | table | gaussdb | 11 MB   | {orientation=row,compression=no,storage_type=USTORE,segment=off}                    | 
(3 rows)
 
gaussdb=# \di+
                                          List of relations
 Schema |      Name       | Type  |  Owner  |  Table   |  Size  |        Storage        | Description 
--------+-----------------+-------+---------+----------+--------+-----------------------+-------------
 public | idx_t_ustore_n1 | index | gaussdb | t_ustore | 37 MB  | {storage_type=USTORE} | 
 public | idx_t_ustore_n2 | index | gaussdb | t_ustore | 118 MB | {storage_type=USTORE} | 
(2 rows)

在有长事务情况下，对表t_ustore update,delete,insert 30次

复制代码 
session1:

\timing
begin;
update t_ustore set owner='A' where object_id=2;

session2:

\timing
set enable_indexscan=off;
set enable_bitmapscan=off;

declare
v_date date;
begin
  for i in 1..30 loop
    select sysdate into v_date;
    update t_ustore set owner='B' where object_id>2; 
    commit;
    delete from t_ustore where object_id>2;
    commit;
    insert into t_ustore select * from test01 where object_id>2 and rownum<=999988;
    commit;
    raise info 'i=%,current_date=%',i,v_date;
  end loop;
end;
/

gaussdb=# \d+
                                                                 List of relations
 Schema |   Name   | Type  |  Owner  |  Size   |                                       Storage                                       | Description 
--------+----------+-------+---------+---------+-------------------------------------------------------------------------------------+-------------
 public | t_ustore | table | gaussdb | 553 MB  | {orientation=row,compression=no,storage_type=USTORE,segment=off,parallel_workers=8} | 
 public | test01   | table | gaussdb | 5614 MB | {orientation=row,compression=no,storage_type=USTORE,segment=off}                    | 
 public | test02   | table | gaussdb | 11 MB   | {orientation=row,compression=no,storage_type=USTORE,segment=off}                    | 
(3 rows)
 
gaussdb=# \di+
                                           List of relations
 Schema |      Name       | Type  |  Owner  |  Table   |  Size   |        Storage        | Description 
--------+-----------------+-------+---------+----------+---------+-----------------------+-------------
 public | idx_t_ustore_n1 | index | gaussdb | t_ustore | 1739 MB | {storage_type=USTORE} | 
 public | idx_t_ustore_n2 | index | gaussdb | t_ustore | 9176 MB | {storage_type=USTORE} | 
(2 rows)

数据库切换到OpenTeleDB，建表的时候存储类型选择XSTORE

复制代码 
postgres=# \d+
                                     List of relations
 Schema |   Name   | Type  |  Owner   | Persistence | Access method |  Size   | Description 
--------+----------+-------+----------+-------------+---------------+---------+-------------
 public | t_xstore | table | postgres | permanent   | xstore        | 132 MB  | 
 public | test     | table | postgres | permanent   | heap          | 12 MB   | 
 public | test01   | table | postgres | permanent   | heap          | 6104 MB | 
(3 rows)
 
postgres=# \di+
                                              List of relations
 Schema |      Name       | Type  |  Owner   |  Table   | Persistence | Access method |  Size  | Description 
--------+-----------------+-------+----------+----------+-------------+---------------+--------+-------------
 public | idx_t_xstore_n1 | index | postgres | t_xstore | permanent   | xbtree        | 49 MB  | 
 public | idx_t_xstore_n2 | index | postgres | t_xstore | permanent   | xbtree        | 128 MB | 
(2 rows)

在有长事务情况下，对表t_xstore update,delete,insert 30次

...省略测试代码...

复制代码 
postgres=# \d+
                                     List of relations
 Schema |   Name   | Type  |  Owner   | Persistence | Access method |  Size   | Description 
--------+----------+-------+----------+-------------+---------------+---------+-------------
 public | t_xstore | table | postgres | permanent   | xstore        | 4077 MB | 
 public | test     | table | postgres | permanent   | heap          | 12 MB   | 
 public | test01   | table | postgres | permanent   | heap          | 6104 MB | 
(3 rows)
 
postgres=# \di+
                                              List of relations
 Schema |      Name       | Type  |  Owner   |  Table   | Persistence | Access method |  Size   | Description 
--------+-----------------+-------+----------+----------+-------------+---------------+---------+-------------
 public | idx_t_xstore_n1 | index | postgres | t_xstore | permanent   | xbtree        | 1397 MB | 
 public | idx_t_xstore_n2 | index | postgres | t_xstore | permanent   | xbtree        | 6548 MB | 
(2 rows)

崖山长事务情况下表膨胀，索引膨胀测试结果：

1.表从116MB到116MB，无膨胀

2.索引1从20MB到20MB，无膨胀

3.索引2从192MB到368MB，膨胀了1.92倍

DM8长事务情况下表膨胀，索引膨胀测试结果：

1.表从110MB到304MB，膨胀了2.76倍

2.索引1从42MB到170MB，膨胀了4.05倍

3.索引2从123MB到776MB，膨胀了6.31倍

GaussDB长事务情况下表膨胀，索引膨胀测试结果：

1.表从126MB到553MB，膨胀了4.39倍

2.索引1从37MB到1739MB，膨胀了47倍

3.索引2从118MB到9176MB，膨胀了77.76倍

OpenTeleDB长事务情况下表膨胀，索引膨胀测试结果：

1.表从132MB到4077MB，膨胀了30.89倍

2.索引1从49MB到1397MB，膨胀了28.51倍

3.索引2从128MB到6548MB，膨胀了51.16倍

有读者可能会有疑问

1.为什么不测试Oracle？其实已经测试过了，限于篇幅，不贴了，结论是表，索引1，索引2无任何膨胀。

2.为什么不测试原生PG? 可以把OpenTeleDB当成原生PG，OpenTeleDB在原生PG上做了XSTORE增强，XSTORE都这么搓，就更别提原生PG了

总结：崖山是真的牛，希望崖山对索引2膨胀再继续优化一下，追上Oracle。

相关推荐
Gauss松鼠会2 天前
【GaussDB】从 sqlplus 到 gsql：Shell 中执行 SQL 文件方案的迁移与改造
数据库·sql·database·gaussdb
晓时谷雨2 天前
达梦数据库适配方案及总结
数据库·达梦·数据迁移
Gauss松鼠会2 天前
【GaussDB】跨用户调用已授权的存储过程，可能会在存储过程内SQL的自定义函数表达式里报错没有权限
数据库·sql·database·gaussdb
云和数据.ChenGuang3 天前
openGauss赋能新能源汽车智能制造——比亚迪MES系统数据库国产化升级案例
数据库·汽车·opengauss·gaussdb·数据库权限管理
BORN（^-^）3 天前
达梦数据库索引删除操作小记
数据库·达梦
云和数据.ChenGuang3 天前
GaussDB 期末考试题与面试题
数据库·opengauss·gaussdb·数据库期末试题
保定公民3 天前
达梦DMDRS数据库同步用户最小权限
数据库·达梦·达梦数据库·数据同步·dm·dmdrs
Neolnfra5 天前
openGauss部署配置指南
数据库·opengauss·gaussdb
DarkAthena6 天前
【GaussDB】跨用户调用已授权的存储过程，可能会在存储过程内SQL的自定义函数表达式里报错没有权限
数据库·sql·gaussdb
热门推荐
01GitHub 镜像站点02Labelme从安装到标注：零基础完整指南03安娜的档案(Anna’s Archive) 镜像网站/国内最新可访问入口（持续更新）04Linux下V2Ray安装配置指南05jdk21下载、安装（Windows、Linux、macOS）06【踩坑笔记】50系显卡适配的 PyTorch 安装07手把手教你通过Gemini3 pro 学生认证，白用一年，手慢无！08Opencode CLI 安装成功,但是启动失败09UV安装并设置国内源10GitLab 零基础入门指南：从安装到项目管理全流程