一个postgresql奇怪慢查询现象的原因和解决

有一个复杂的CTE查询，原来最后一步直接查询的语句如下

sql 复制代码

select t.rn ,s from (select rn,s,row_number()over(partition by rn order by rn)resn from t where t.next_pos=0)t,b WHERE resn=1 and t.rn=b.rn;
...
Time: 1357.870 ms (00:01.358)

想统计最终结果中表t的行数，而不是关联后的行数，于是用下面的查询

sql 复制代码

select count(*)from t where t.next_pos=0;  -- 2.754秒
 count
-------
  2843
(1 row)

Time: 2754.143 ms (00:02.754)

开始以为是row_number()函数过滤的影响，结果并不是，还是比较慢

sql 复制代码

select count(*)from (select rn,s,row_number()over(partition by rn order by rn)resn from t where t.next_pos=0)t1 WHERE resn=1; --2.855秒

 count
-------
  1000
(1 row)

Time: 2855.320 ms (00:02.855)

而原来未加count( *)的直接查询只要一半的时间。奇怪的是，对加了表连接的结果再count( *)反而更快，

sql 复制代码

select count(*)from (select rn,s,row_number()over(partition by rn order by rn)resn from t where t.next_pos=0)t1, b WHERE resn=1 and t1.rn=b.rn ; --1.239秒

 count
-------
  1000
(1 row)

Time: 1239.115 ms (00:01.239)

查看执行计划，慢查询里有一处耗时的操作它的关联条件很长。

复制代码

 Nested Loop  (cost=0.00..20184.70 rows=1550 width=142) (actual time=0.038..1595.842 rows=45778.00 loops=1)
                             Join Filter: (substr(CASE CASE WHEN (length(replace(substr(replace(replace((s.b)::text, '
 '::text, ''::text), ' '::text, ''::text), hh), '?'::text, ''::text)) > length(replace(substr(replace(replace((s.b)::text, '
 '::text, ''::text), ' '::text, ''::text), ss), '?'::text, ''::text))) THEN 0 ELSE 1 END WHEN 0 THEN replace(replace((s.b)::text, '
 '::text, ''::text), ' '::text, ''::text) ELSE reverse(replace(replace((s.b)::text, '
 '::text, ''::text), ' '::text, ''::text)) END, all_pos.pos, 1) = (all_pos.n)::text)

更快的语句执行计划同样的地方就只有 (substr(b_1.b, all_pos.pos, 1) = (all_pos.n)::text)。成本也只有一半

复制代码

                      ->  Nested Loop  (cost=0.00..9316.20 rows=1550 width=84) (actual time=0.006..183.893 rows=45778.00 loops=1)
                             Join Filter: (substr(b_1.b, all_pos.pos, 1) = (all_pos.n)::text)

而上述过滤条件是在另两个子查询中

sql 复制代码

a as(select rn, replace(replace(b,chr(10),''),' ','')b from s
,
b  as 
(select rn,case flag when 0 then b else reverse(b) end b,flag from(select rn, b ,case when length(replace(substr(b,hh),'?',''))>length(replace(substr(b,ss),'?','')) then 0 else 1 end flag from a
)s),

看来这几个子查询都没有实体化，而是展开到了 Nested Loop连接的句子中

给子查询b 加了个 MATERIALIZED，变成b as MATERIALIZED (select rn,case flag when ... , 所有的count就都快了。

sql 复制代码

select count(*)from t where t.next_pos=0;
 count
-------
  2843
(1 row)

Time: 1249.923 ms (00:01.250)

select count(*)from (select rn,s,row_number()over(partition by rn order by rn)resn from t where t.next_pos=0)t1 WHERE resn=1; 

 count
-------
  1000
(1 row)

Time: 1218.167 ms (00:01.218)