union与in结合用法优化

1、问题

最近项目中看到这种写法

bash 复制代码

select count(1) from ( with ss as
             (select id from t1 where (id='A1' or oid='A2')
             union 
             select oid from t1 where (id='A1' or oid='A2')
             )
       select * 
         from t2 
        where (t2.mid in (select id from ss) or t2.mid in (select ipmid 
                        from t3
                       where mid in (select id from ss) 
                         and status=1) ) 
          and t2.pcode='03' 
          )

计划：

从计划上看生成两个HEAP TABLE，其中一个是来自with ss中的，另外一个是t2主查询的。

我们可以理解到or条件会提取公共部分生成heap table，减少扫描次数，但它会让条件没有先过滤，从而出现CSCN（全表扫描）引起性能问题，优化关键在于如何避免全表扫。

从语句条件(id='A1' or oid='A2')知道过滤性较好，可以使用索引，所以要优化可以将or条件改写，之前提过可以改成union all。

第一部分

bash 复制代码

select id from t1 where (id='A1' or oid='A2')
             union 
             select oid from t1 where (id='A1' or oid='A2')
这个很简单可以改写成
select id from t1 where id='A1'
             union 
             select id from t1 where oid='A2'
             union 
             select oid from t1 where id='A1'
             union 
             select oid from t1 where oid='A2'

第二部分

bash 复制代码

select * 
         from t2 
        where (t2.mid in (select id from ss) or t2.mid in (select ipmid 
                        from t3
                       where mid in (select id from ss) 
                         and status=1) ) 
这个我们可以注意到or条件t2.mid in是一样的，这也是为什么会生成HEAP TABLE的原因，那么可以将in里面的先union all再与主查询做in
改写成
select * 
         from t2 
        where t2.mid in (select id from ss
              union all 
              select ipmid from t3 where mid in (select id from ss) and status=1 )

基本上改写完成，这里还要一点说明，in子查询是有隐性去重的效果，我们知道in (1,2,2)等价于in (1,2),无论in集合里是否重复都对主查询结果没影响，那么应该减少去重操作。所以第一部分所有的union可以改成union all。

2、改写

bash 复制代码

select count(1) 
  from (with ss as
             (select id from t1 where id='A1'
             union all
             select id from t1 where oid='A2'
             union all 
             select oid from t1 where id='A1'
             union all
             select oid from t1 where oid='A2' 
             )
       select * 
         from t2 
        where t2.mid in (select id from ss
              union all 
              select ipmid from t3 where mid in (select id from ss) and status=1 )
          and t2.pcode='03' )

计划：

从计划上看，避免了全表扫描，执行效率提升几十倍。

3、小结

1、在得知in子查询能过滤出较少数据时，形如(c1 in or c1 in) 可以改写成c1 in (select union all select)。

2、 In 子查询本身有隐性去重中union应改写成union all，但要注意一点除非in子查询里面的重复值本身太多（即集合数量过多）不利于与主查询关联。

4、附加测试数据

bash 复制代码

create table t1 (id varchar2(20) primary key,oid varchar2(20),status int);
insert into t1 select 'A'||level,'A'||to_char(round(dbms_random.value(1,10000),0)),round(dbms_random.value(1,5),0)
from dual connect by level<=100000;
commit;
dbms_stats.gather_table_stats(USER,'T1',null,100);
drop table if exists t2;
create table t2 (id varchar2(36) primary key,mid varchar2(20),is_d int,pcode varchar2(2),status int);
insert into t2 select sys_guid(),'A'||to_char(round(dbms_random.value(1,10000),0)),round(dbms_random.value(1,5),0),'03',round(dbms_random.value(1,5),0)
from dual connect by level<=400000;
commit;
dbms_stats.gather_table_stats(USER,'T2',null,100);
drop table if exists t3;
create table t3 (id varchar2(36) primary key,ipmid varchar2(20),mid varchar2(20),status int);
insert into t3 select sys_guid(),'A'||to_char(round(dbms_random.value(1,10000),0)),'A'||to_char(round(dbms_random.value(1,10000),0)),round(dbms_random.value(1,5),0)
from dual connect by level<=300000;
commit;
dbms_stats.gather_table_stats(USER,'T3',null,100);

create index IDX_DM_t1_oid on t1(oid);
create index IDX_T2_MID on T2(MID);
create index IDX_T3_MID on T3(MID);