最新发现公司的数据库TempDb存储经常飙升到很高,后来定位后发现是因为Service Broker调用的时候没有将conversation结束掉,导致表记录一直在sys.conversation_endpoints。
far_service是service broker对应的service名称。
你可以在你自己的数据库-->Service Broker -->Services中找到你创建的service
sql
select top 100 ce.state_desc,ce.far_service,ce.security_timestamp
from
sys.conversation_endpoints ce
left join sys.services ss on ce.service_id=ss.service_id
where state_desc='CONVERSING'
and security_timestamp>'2025-11-07'
order by security_timestamp desc
以下内容具体讲解怎么样启用和创建service broker。
本例子是用Service Broker插入数据到AuditTrail中,因为AuditTrail用于记录所有用户的操作日志,每天会产生大量的日志记录,如果实时插入会影响系统的系统,而采用Service Broker就是为了实现异步插入以换件数据库的负载。
目录
[1. Service Broker的启用](#1. Service Broker的启用)
[2. 创建Queue](#2. 创建Queue)
[3. 创建Service](#3. 创建Service)
[4. 创建测试表AuditTrail,记录执行失败的记录表AuditTrailServiceBrokerException和用户自定义表类型dtAuditTrail](#4. 创建测试表AuditTrail,记录执行失败的记录表AuditTrailServiceBrokerException和用户自定义表类型dtAuditTrail)
[5. 创建调用Service Broker的存储过程SP_Service_Broker_AuditTrail](#5. 创建调用Service Broker的存储过程SP_Service_Broker_AuditTrail)
[6. 创建调用 AuditTrailService的存储过程](#6. 创建调用 AuditTrailService的存储过程)
[7. 启用SQL Agent](#7. 启用SQL Agent)
[8. 创建SQL Job (Service_Broker_AuditTrail)调用存储过程SP_Service_Broker_AuditTrail](#8. 创建SQL Job (Service_Broker_AuditTrail)调用存储过程SP_Service_Broker_AuditTrail)
[9. 测试](#9. 测试)
1. Service Broker的启用
数据库默认是不启用Service Broker的,而且当你启用了Service Boker后如果重新还原新的数据库,Service Broker会被禁用,需要你重新启用它。
可以通过以下命令查看是否启用:
sql
-- 需要将YourDatabaseName改成你自己的数据库,注意此处不可以用[YourDatabaseName]
-- s_broker_enabled = 0 表示Service Broker没有启用;1 表示启用
SELECT is_broker_enabled FROM sys.databases WHERE name = 'YourDatabaseName';
如果返回结果显示为0,表示未启用,可以通过以下脚本启用:
执行时间依赖于你数据库的大小,200G数据库执行时间大概5分钟。
sql
-- 启用Service Broker。 YourDatabaseName需要更改成你自己的数据库名
ALTER DATABASE [YourDatabaseName] SET ENABLE_BROKER;
执行成功后,可用第一个脚本进行验证,启用成功后,返回结果为1.
2. 创建Queue
运行一下命令进行Queue的创建。创建成功后,可依次展开Database--》你的数据库--》Service Broker --> Queues。
sql
-- 更改YourDatabaseName为你自己的数据库名
USE [YourDatabaseName]
GO
--创建Queue (AuditTrailSyncQueue是我本地数据库创建的例子,你可以更该为你自己的,后面创建Service的时候会用到)
CREATE QUEUE [dbo].[AuditTrailSyncQueue] WITH STATUS = ON , RETENTION = OFF , POISON_MESSAGE_HANDLING (STATUS = ON)
后续的操作记录会被记录到此表中,等Job完成后记录会自动删除。
sql
--可以查看当前还有多少记录等待Service Broker处理
select * from [dbo].[AuditTrailSyncQueue]
3. 创建Service
运行以下命令基于创建的Queue创建Service。
可依次展开Database--》你的数据库--》Service Broker --> Services.
sql
USE [YourDatabaseName]
GO
CREATE SERVICE [AuditTrailService] ON QUEUE [dbo].[AuditTrailSyncQueue]
4. 创建测试表AuditTrail,记录执行失败的记录表AuditTrailServiceBrokerException和用户自定义表类型dtAuditTrail
测试表AuditTrail
sql
CREATE TABLE [dbo].[AuditTrail](
[Id] [bigint] IDENTITY(1,1) NOT NULL ,
[TableRowId] [int] NOT NULL,
[TableName] [varchar](100) NOT NULL,
[Action] [varchar](max) NOT NULL,
[Serialization] [varchar](max) NOT NULL,
[Username] [varchar](max) NOT NULL,
[ChangeDate] [datetime] NOT NULL,
CONSTRAINT [PK_AuditTrail] PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
Service Broker执行失败的记录表
sql
CREATE TABLE [dbo].[AuditTrailServiceBrokerException](
[Id] [int] IDENTITY(1,1) NOT NULL,
[Message] [xml] NULL,
[ErrorMessage] [varchar](500) NULL,
[DateOccured] [datetime] NULL,
PRIMARY KEY CLUSTERED
(
[Id] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, IGNORE_DUP_KEY = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
) ON [PRIMARY] TEXTIMAGE_ON [PRIMARY]
GO
用户自定义表类型dtAuditTrail, 用于数据间的转换。
sql
CREATE TYPE [dbo].[dtAuditTrail] AS TABLE(
[id] [bigint] NULL,
[tablename] [varchar](100) NULL,
[action] [varchar](10) NULL,
[serialization] [nvarchar](max) NULL,
[executeBy] [varchar](2000) NULL,
[execdate] [datetime] NULL
)
GO
5. 创建调用Service Broker的存储过程SP_Service_Broker_AuditTrail
创建存储过程用于获取Service Broker的Queue中的记录,进行后续数据的操作。
sql
CREATE PROCEDURE SP_Service_Broker_AuditTrail
AS
BEGIN
DECLARE @Handle UNIQUEIDENTIFIER;
DECLARE @MessageType SYSNAME;
DECLARE @Message XML
BEGIN TRY
SET XACT_ABORT ON
WAITFOR (
RECEIVE TOP (1)
@Handle = conversation_handle,
@MessageType = message_type_name,
@Message = message_body
FROM dbo.[AuditTrailSyncQueue]--如果你上面更改了Queue的名字,此处需要改成你自己的Queue
), TIMEOUT 1000
-- exit if no message received (timeout)
IF @Handle IS NULL BEGIN
SET XACT_ABORT OFF;
RETURN;
END
IF @Message IS NOT NULL BEGIN
INSERT INTO AuditTrail
SELECT
id = Node.Data.value('id[1]', 'INT'),
tablename = Node.Data.value('tablename[1]', 'NVARCHAR(100)'),
action = Node.Data.value('action[1]', 'NVARCHAR(20)'),
serialization = isnull(replace(replace(Node.Data.value('serialization[1]', 'NVARCHAR(max)'), '1lthan1', '<'), '2gthan2', '>'),''),
executeBy = Node.Data.value('executeBy[1]', 'NVARCHAR(2000)'),
execdate = Node.Data.value('execdate[1]', 'datetime')
FROM @Message.nodes('//mydata') Node(Data)
--select @@IDENTITY
END
END CONVERSATION @Handle;
END TRY
BEGIN CATCH
END CONVERSATION @Handle;
declare @ERROR VARCHAR(500)
DECLARE @dt DATEtime =GETDATE()
SET @ERROR =ERROR_MESSAGE()
-- 记录执行失败后的错误信息
INSERT INTO dbo.AuditTrailServiceBrokerException
VALUES (
@Message,
@ERROR ,
@dt
);
declare @messagebody varchar(5000) = concat('<b>Error in processing AuditTrail Service Broker Queue</b><BR><b>Please check the error Message field</b><BR> <b>ERROR:</b>',isnull(@ERROR, 'Error cant be Determined'))
-- 执行失败后,发送异常邮件到DBA
exec msdb.dbo.sp_send_dbmail
@profile_name = 'DBAMail',
@recipients = '18901599114@163.com',-- 更改为你们DB-Admin的邮箱,用于在Service Broker执行失败的时候发送邮件
@subject = 'AuditTrail Serialization: Error in processing AuditTrail Service Broker Queue',
@body = @messagebody,
@body_format = 'HTML'
END CATCH;
SET XACT_ABORT OFF
END
6. 创建调用 AuditTrailService的存储过程
此存储过程调用 AuditTrailService会插入一条记录到AuditTrailSyncQueue中。
sql
CREATE PROC [dbo].[SP_INSERT_AUDITTRAIL] @input dbo.[dtAuditTrail] READONLY
AS
BEGIN
IF NOT EXISTS (
SELECT *
FROM @input
)
RETURN
DECLARE @XMLMessage XML
DECLARE @Handle UNIQUEIDENTIFIER;
DECLARE @id INT
DECLARE @tablename VARCHAR(200)
DECLARE @serialization XML
DECLARE @sql VARCHAR(2000)
SELECT * INTO #t1 from @input where isnull(serialization,'')<>'' and tablename not in ('AccrualDetailsSnapshot', 'NoticeLogs')
/**GO TO THIS IF LOOP IF SERIALIZATION IS NULL OR BLANK VALUE**/
IF EXISTS (SELECT 1 FROM @input WHERE ISNULL(serialization,'')='')
BEGIN
SELECT *, CAST(0 AS BIT) AS isprocessed INTO #t2 FROM @input WHERE isnull(serialization,'') =''
WHILE EXISTS(SELECT 1 FROM #t2 WHERE isprocessed=0)
BEGIN
SELECT TOP 1 @id = id
,@tablename = tablename
,@serialization = REPLACE(REPLACE(serialization, '<', '1lthan1'), '>', '2gthan2')
FROM #t2
WHERE isprocessed=0
CREATE TABLE #t(serialization XML)
SET @sql='DECLARE @serialization xml;SET @serialization = (SELECT * FROM '+@tablename+' WHERE id = '+ltrim(@id)+' for xml auto); INSERT INTO #t SELECT @serialization'
EXEC(@sql)
SELECT @serialization=serialization FROM #t
DROP TABLE #t
INSERT INTO #t1
SELECT id
,tablename
,action
,replace(replace(cast(@serialization as varchar(max)), '<', '1lthan1'), '>', '2gthan2') AS serialization
,executeBy
,execdate
FROM @input WHERE id=@id and tablename=@tablename
UPDATE #t2
SET isprocessed=1
WHERE id=@id and tablename=@tablename
END
END
IF NOT EXISTS (SELECT 1 FROM #t1)
BEGIN
RETURN
END
SELECT @XMLMessage = (
SELECT isnull(id,999) as id
,isnull(tablename,'unknown') as tablename
,isnull(action,'unknown') as action
,isnull(serialization,'unknown') as serialization
,isnull(executeBy,'unkown') as executeBy
,isnull(execdate,getdate()) as execdate
FROM #t1
FOR XML PATH('mydata')
,TYPE
);
BEGIN DIALOG CONVERSATION @Handle
FROM SERVICE AuditTrailService TO SERVICE 'AuditTrailService' ON CONTRACT AuditTrailMessages
WITH ENCRYPTION = OFF;
SEND ON CONVERSATION @Handle MESSAGE TYPE AuditTrailSync(@XMLMessage);
IF OBJECT_ID('tempdb..#t1') IS NOT NULL DROP TABLE #t1
IF OBJECT_ID('tempdb..#t2') IS NOT NULL DROP TABLE #t2
END
7. 启用SQL Agent
确保你的SQL Server安装了SQL Agent,如果没有安装需要通过安装包进行增量安装,安装方法自行AI。
打开Services,找到SQL Server Agent,右击启用它,并设置Automatically Start,或者Start Delay.
8. 创建SQL Job (Service_Broker_AuditTrail)调用存储过程SP_Service_Broker_AuditTrail
可通过以下脚本创建Job,调用SP_Service_Broker_AuditTrail
sql
-- 创建JOB 调用, 数据库目标YourDatabaseName, 需要改为你自己的数据库名
USE [msdb]
GO
/****** Object: Job [Service_Broker_AuditTrail] Script Date: 11/7/2025 4:40:46 PM ******/
BEGIN TRANSACTION
DECLARE @ReturnCode INT
SELECT @ReturnCode = 0
/****** Object: JobCategory [[Uncategorized (Local)]] Script Date: 11/7/2025 4:40:46 PM ******/
IF NOT EXISTS (SELECT name FROM msdb.dbo.syscategories WHERE name=N'[Uncategorized (Local)]' AND category_class=1)
BEGIN
EXEC @ReturnCode = msdb.dbo.sp_add_category @class=N'JOB', @type=N'LOCAL', @name=N'[Uncategorized (Local)]'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
END
DECLARE @jobId BINARY(16)
EXEC @ReturnCode = msdb.dbo.sp_add_job @job_name=N'Service_Broker_AuditTrail',
@enabled=1,
@notify_level_eventlog=0,
@notify_level_email=0,
@notify_level_netsend=0,
@notify_level_page=0,
@delete_level=0,
@description=N'No description available.',
@category_name=N'[Uncategorized (Local)]',
@owner_login_name=N'zwang.devsql', @job_id = @jobId OUTPUT
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
/****** Object: Step [Service_Broker_AuditTrail] Script Date: 11/7/2025 4:40:46 PM ******/
EXEC @ReturnCode = msdb.dbo.sp_add_jobstep @job_id=@jobId, @step_name=N'Service_Broker_AuditTrail',
@step_id=1,
@cmdexec_success_code=0,
@on_success_action=1,
@on_success_step_id=0,
@on_fail_action=3,
@on_fail_step_id=0,
@retry_attempts=0,
@retry_interval=0,
@os_run_priority=0, @subsystem=N'TSQL',
@command=N'exec SP_Service_Broker_AuditTrail', -- 调用存储过程SP_Service_Broker_AuditTrail
@database_name=N'YourDatabaseName', --需要将YourDatabaseName改为你自己的数据库名
@flags=4
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_update_job @job_id = @jobId, @start_step_id = 1
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobschedule @job_id=@jobId, @name=N'Continuous',
@enabled=1,
@freq_type=4,
@freq_interval=1,
@freq_subday_type=2,
@freq_subday_interval=10,
@freq_relative_interval=0,
@freq_recurrence_factor=0,
@active_start_date=20220711,
@active_end_date=99991231,
@active_start_time=0,
@active_end_time=235959,
@schedule_uid=N'78165068-0862-453c-9b79-f4f8e5070e3f'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
EXEC @ReturnCode = msdb.dbo.sp_add_jobserver @job_id = @jobId, @server_name = N'(local)'
IF (@@ERROR <> 0 OR @ReturnCode <> 0) GOTO QuitWithRollback
COMMIT TRANSACTION
GOTO EndSave
QuitWithRollback:
IF (@@TRANCOUNT > 0) ROLLBACK TRANSACTION
EndSave:
GO
9. 测试
可以先禁用Service_Broker_AuditTrail job,然后运行以下脚本进行数据测试。
主要是调用SP_INSERT_AUDITTRAIL去调用Service。
sql
declare @UserName nvarchar(500)='Test-By-Evan'
DECLARE @AuditTrail AS [dtAuditTrail]
declare @TestDate datetime=GETDATE()
INSERT INTO @AuditTrail
SELECT 1,
TableName='AuditTrail',
[action]='INSERT',
Serialization='',
ExecutedBy=@UserName
,execdate= @TestDate
EXEC SP_INSERT_AUDITTRAIL @AuditTrail
此时可以通过以下脚本查询到有一条新的记录插入到sys.conversation_endpoints中。
sql
select top 2 ce.state_desc,ce.far_service,ce.security_timestamp
from
sys.conversation_endpoints ce
left join sys.services ss on ce.service_id=ss.service_id
where state_desc='CONVERSING'
and security_timestamp>'2025-11-07'--改成你的时间,注意是UTC时间
order by security_timestamp desc
然后启用SQL Job,执行成功后,上面的记录会被删除,然后AuditTrail表中插入新的记录。
sql
-- 插入记录
SELECT TOP 5 * FROM AuditTrail WITH (NOLOCK)
WHERE TableRowId =1 and TableName='AuditTrail'
--and Username=@UserName
order by id desc
--此时以下记录返回值中没有刚才插入的记录了
select top 2 ce.state_desc,ce.far_service,ce.security_timestamp
from
sys.conversation_endpoints ce
left join sys.services ss on ce.service_id=ss.service_id
where state_desc='CONVERSING'
and security_timestamp>'2025-11-07'
order by security_timestamp desc