上午遇到一个情况，通过SCAN监听访问数据库时候，长时间无反应，大约2、3分钟后报错丢失连接ORA-12547: TNS:lost contact ;
常规思路下，遇到次问题，有一系列的排查、排除方向，去定位和处理问题。
本次的情况，按照对系统的了解，结合N年的DBA经验，快速定位问题并解决；过程如下：
1、在这个Oracle Exadata环境，有多个数据库，未有人反馈问题，初步判断大部分系统是正常的。看集群状态，运行正常；基本的磁盘空间、TOP命令输出均正常。
2、SCAN监听问题，用了2、3个数据库的SERVICE_NAME去连，都有问题，判断为SCAN监听问题，不是个别数据库问题。
3、用VIP进行连接，连接、查询均正常。
4、查看SCAN 监听状态，lsnrctl status LISTENER_SCAN1执行后发现需要很长时间才可以显示；里面数据库服务名状态都是READY,仔细看Uptime,说明刚重启过，Uptime 0 days 0 hr. 1 min. 5 sec，
5、查看SCAN监听程序日志大小，特别多；直觉反映应该是连接频繁，用监听日志分析的命令 awr系列命令，发现没问题；直接打开监听日志看几千行，问题就清楚了，有大量的同一个IP地址（IP已经脱敏）的连接信息报错--无SID，连接信息是无效的：29-FEB-2024 10:14:57 * <unknown connect data> * (ADDRESS=(PROTOCOL=tcp)(HOST=192.26.32.193)(PORT=42454)) * establish * <unknown sid> *，大量连接导致监听程序异常：
29-FEB-2024 10:13:50 * 12540
TNS-12540: TNS:internal limit restriction exceeded
29-FEB-2024 10:13:50 * 12582
TNS-12582: TNS:invalid operation
TNS-12615: TNS:preempt error
29-FEB-2024 10:13:50 * 1159
TNS-01159: Internal connection limit has been reached; listener has shut down
TNS-12540: TNS:internal limit restriction exceeded
6、把IP发给对应软件公司，确定为一台备份机；停止程序后，监听日志查看正常不再有高频连接，此后SCAN监听未发生过重启，系统恢复正常。
对应的排查国产及日志如下：
1、监听程序异常时候的日志：
[grid@hahaha trace]$ tail -f listener_scan1.log
9-FEB-2024 10:13:26 * service_update * czqcglxt2 * 0
Thu Feb 29 10:13:34 2024
29-FEB-2024 10:13:34 * service_update * less1 * 0
29-FEB-2024 10:13:34 * service_update * hzwsali2 * 0
Thu Feb 29 10:13:50 2024
29-FEB-2024 10:13:50 * <unknown connect data> * (ADDRESS=(PROTOCOL=tcp)(HOST=192.1.1.1)(PORT=39854)) * establish * <unknown sid> * 12525
TNS-12525: TNS:listener has not received client's request in time allowed
TNS-12535: TNS:operation timed out
TNS-12606: TNS: Application timeout occurred
29-FEB-2024 10:13:50 * <unknown connect data> * (ADDRESS=(PROTOCOL=tcp)(HOST=192.26.32.193)(PORT=39856)) * establish * <unknown sid> * 12525
TNS-12525: TNS:listener has not received client's request in time allowed
TNS-12535: TNS:operation timed out
TNS-12606: TNS: Application timeout occurred
29-FEB-2024 10:13:50 * 12540
TNS-12540: TNS:internal limit restriction exceeded
29-FEB-2024 10:13:50 * 12582
TNS-12582: TNS:invalid operation
TNS-12615: TNS:preempt error
29-FEB-2024 10:13:50 * 1159
TNS-01159: Internal connection limit has been reached; listener has shut down
TNS-12540: TNS:internal limit restriction exceeded
No longer listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.1)(PORT=1521)))
No longer listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
Thu Feb 29 10:13:51 2024
System parameter file is /u01/app/11.2.0.4/grid/network/admin/listener.ora
Log messages written to /u01/app/11.2.0.4/grid/log/diag/tnslsnr/haha/listener_scan1/alert/log.xml
Trace information written to /u01/app/11.2.0.4/grid/log/diag/tnslsnr/haha/listener_scan1/trace/ora_211518_140567299258112.trc
Trace level is currently 0
Started with pid=211518
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=LISTENER_SCAN1)))
Listener completed notification to CRS on start
TIMESTAMP * CONNECT DATA [* PROTOCOL INFO] * EVENT [* SID] * RETURN CODE
29-FEB-2024 10:13:51 * (CONNECT_DATA=(CID=(PROGRAM=)(HOST=hahahaha)(USER=grid))(COMMAND=status)(ARGUMENTS=64)(SERVICE=LISTENER_SCAN1)(VERSION=186647552)) * status * 0
29-FEB-2024 10:13:51 * version * 0
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=10.1.1.1)(PORT=1521)))
29-FEB-2024 10:13:52 * service_register * LsnrAgt * 0
29-FEB-2024 10:13:52 * service_register * lxxxxxxgl1 * 0
............
29-FEB-2024 10:09:49 * <unknown connect data> * (ADDRESS=(PROTOCOL=tcp)(HOST=192.1.1.1)(PORT=60072)) * establish * <unknown sid> * 12525
TNS-12525: TNS:listener has not received client's request in time allowed
TNS-12535: TNS:operation timed out
TNS-12606: TNS: Application timeout occurred
29-FEB-2024 10:09:49 * <unknown connect data> * (ADDRESS=(PROTOCOL=tcp)(HOST=192.1.1.1)(PORT=60074)) * establish * <unknown sid> * 12525
TNS-12525: TNS:listener has not received client's request in time allowed
TNS-12535: TNS:operation timed out
TNS-12606: TNS: Application timeout occurred
2、查看确认一下对应IP的高频率连接问题
情况监听日志，几分钟后的查看
[grid@HAHA trace]$ cat listener_scan1.log |grep 192.1.1.1|wc -l
19329
3、根据有问题的IP，找软件维护人员查看，关闭对应程序后，系统恢复。