MediaStream使用webRtc多窗口传递

web前端进阶者2024-04-23 15:50

最近在做音视频通话，有个需求是把当前会话弄到另一个窗口单独展示，但是会话是属于主窗口的，多窗口通信目前不能直接传递对象，所以想着使用webRtc在主窗口和兄弟窗口建立连接，把主窗口建立会话得到的MediaStream传递给兄弟窗口;

主窗口界面
兄弟窗口界面
主窗口点击发送本地媒体后
呼叫封装_主窗口
typescript 复制代码 
// 主窗口WebRtc_呼叫
class CallWindowWebRtc {
    // 广播通道
    curBroadcas: BroadcastChannel;
    // webRtc点对点连接
    peerConnection: RTCPeerConnection;
    // 广播通道
    constructor({broadcastChannelName = 'yyh_text'}) {
        this.curBroadcas = CreateBroadcastChannel(broadcastChannelName);
        this.curBroadcas.onmessage = (event) => this.onMessage(event);
        // 处理页面刷新和关闭方法
        this.handlePageRefreshClose();
    }

    // 接收消息
    onMessage(event: any) {
        const msg = event.data;

        // 收到远端接听消息
        if (msg.type === 'answer') {
            this.handleSedRemoteSDP(msg);
        }
        if (msg.type === 'hangup') {
            this.hangup();
        }
    }
    // 发送消息_方法
    postMessage(msg: any) {
        this.curBroadcas.postMessage(msg);
    }
    // 处理页面刷新和关闭方法
    handlePageRefreshClose() {
        window.addEventListener('beforeunload', () => {
            this.postMessage({data: {event: 'mainPageRefresh', eventName: '主页面刷新了'}});
        });
        window.addEventListener('beforeunload', () => {
            this.postMessage({data: {event: 'mainPageClose', eventName: '主页面关闭了'}});
        });
    }
    // 处理媒体停止
    handleStreamStop() {
        if (!this.peerConnection) { return; }
        let localStream = this.peerConnection.getSenders();
        localStream.forEach((item: any) => {
            item.track.stop();
        })
    }
    // 卸载
    unmount() {
        if (this.peerConnection) {
            let localStream = this.peerConnection.getSenders();
            localStream.forEach((item: any) => {
                item.track.stop();
            })
            this.peerConnection.close();
            this.peerConnection.onicecandidate = null;
            this.peerConnection.ontrack = null;
            this.peerConnection.oniceconnectionstatechange = null;
            this.peerConnection = null;
        }
        if (this.curBroadcas) {
            this.curBroadcas.onmessage = null;
            this.curBroadcas = null;
        }
    }
    // ICE连接状态回调
    handleOniceconnectionstatechange(event) {
        // 1.检查网络配置
        if (this.peerConnection.iceConnectionState === 'checking') {
            // 发送订阅消息_给外部
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'checking', eventName: '检查网络配置'});
            // 2.ICE候选者被交换并成功建立了数据传输通道
        } else if (this.peerConnection.iceConnectionState === 'connected') {
            // 发送订阅消息_给外部
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'connected', eventName: 'ICE候选者被交换并成功建立了数据传输通道'});
            // 3.当连接被关闭或由于某种原因（如网络故障、对端关闭连接等）中断时
        } else if (this.peerConnection.iceConnectionState === 'disconnected') {
            this.hangup();
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'connected', eventName: 'ICE接被关闭或由于某种原因断开'});
        };
    }
    // 发送订阅消息给外部
    onSubscriptionMsg(msg: {}) {
    }
    // 创建全新的 RTCPeerConnection
    handleCreateNewPerrc() {
        // 停止媒体 
        this.handleStreamStop();
        // 最好每一次通话都单独创建一个RTCPeerConnection对象,防止复用导致ICE候选的收集受到之前连接的影响,导致画面延迟加载,或其它异常问题无法排查处理;
        this.peerConnection = new RTCPeerConnection();
        this.peerConnection.onicecandidate = (event) => this.onIcecandidate(event);
        this.peerConnection.ontrack = (event) => this.handleOnTrack(event);
        this.peerConnection.oniceconnectionstatechange  = (event) => this.handleOniceconnectionstatechange(event);
    }
    // 呼叫
    call(stream?: MediaStream) {
        return new Promise((resolve, reject) => {
            this.handleCreateNewPerrc();
            this.handleStreamAddPeerConnection(stream);
            this.handleCreateOffer().then((offer) => {
                // 存入本地offer
                this.handleLocalDes(offer).then(() => {
                    // 给远端发sdp
                    this.handleSendSDP();
                    resolve({code: 1, message: '发送sdp给远端'});
                }).catch(() => {
                    reject({code: 0, message: '存入本地offer失败'});
                });
            }).catch(() => {
                reject({code: 0, message: '创建offer失败'});
            });
        });
    }
    // 挂断
    hangup() {
        if (!this.peerConnection) {
            return;
        }
        if (this.peerConnection.signalingState === 'closed') {
            return;
        };
        this.postMessage({type: 'hangup'});
        // 停止媒体流
        let localStream = this.peerConnection.getSenders();
        localStream.forEach((item: any) => {
            item.track.stop();
        });
        // 关闭peerConection
        this.peerConnection.close();

    }
    // 1.获取本地媒体流
    getUserMediaToStream(audio: true, video: true) {
        return navigator.mediaDevices.getUserMedia({audio, video});
    }
    // 2.把媒体流轨道添加到 this.peerConnection 中
    handleStreamAddPeerConnection(stream?: MediaStream) {
        if (!stream) {
            stream = new MediaStream();
        }
        const tmpStream = new MediaStream();
        const audioTracks = stream.getAudioTracks();
        const videoTracks = stream.getVideoTracks();
        if (audioTracks.length) {
            tmpStream.addTrack(audioTracks[0]);
            this.peerConnection.addTrack(tmpStream.getAudioTracks()[0], tmpStream);
        }
        if (videoTracks.length) {
            tmpStream.addTrack(videoTracks[0]);
            this.peerConnection.addTrack(tmpStream.getVideoTracks()[0], tmpStream);
        }
    }
    // 3.创建createOffer
    handleCreateOffer() {
        return this.peerConnection.createOffer();
    }
    // 4.设置本地SDP描述
    handleLocalDes(offer) {
        return this.peerConnection.setLocalDescription(offer);
    }
    // 5.发送SDP消息给远端
    handleSendSDP() {
        if (this.peerConnection.signalingState === 'have-local-offer') {
            // 使用某种方式将offer传递给窗口B
            const answerData = {
                type: this.peerConnection.localDescription.type,
                sdp: this.peerConnection.localDescription.sdp
            };
            this.curBroadcas.postMessage(answerData);
        }
    }
    // 6.收到远端接听_存远端SDP
    handleSedRemoteSDP(msg: any) {
        // if (this.peerConnection.signalingState === 'stable') { return; }
        const answerData = msg;
        const answer = new RTCSessionDescription(answerData);
        return this.peerConnection.setRemoteDescription(answer);
    }
    // 7.用于处理ICE
    onIcecandidate(event) {
        // 如果event.candidate存在，说明有一个新的ICE候选地址产生了
        if (event.candidate) {  
            // 将ICE候选地址, 通常需要通过信令服务器发送给对端
            this.curBroadcas.postMessage({type: 'candidate', candidate: JSON.stringify(event.candidate)});  
        } else {  
            // 如果event.candidate不存在，则表示所有候选地址都已经收集完毕
            // 在某些情况下，这可能意味着ICE候选过程已完成，但并非总是如此
            // 因为在某些情况下，会有多轮ICE候选生成
        }
    }
    // 8.监听轨道赋值给video标签onTrack
    handleOnTrack(event: any) {
        let remoteStream = event.streams[0];
        // 发送订阅消息_给外部
        this.onSubscriptionMsg({event: 'remoteStreams', eventName: '远端视频准备好了', remoteStream})
    }
    
}
兄弟窗口封装
typescript 复制代码 
// 其它窗口WebRtc_接听
class AnswerWindowWebRtc {
    // 广播通道
    curBroadcas: BroadcastChannel;
    // webRtc点对点连接
    peerConnection: RTCPeerConnection;
    
    constructor({broadcastChannelName = 'yyh_text'}) {
        this.curBroadcas = CreateBroadcastChannel(broadcastChannelName);
        this.curBroadcas.onmessage = (event) => this.onMessage(event);
        this.handlePageRefreshClose();
    }

    
    // 接收消息
    onMessage(event: any) {
        const msg = event.data;
        // 收到远端SDP
        if (msg.type === 'offer') {
            this.handleCreateNewPerrc();
            this.onSubscriptionMsg({event: 'incomingCall', eventName: '收到新的来电', offer: msg});
        }
        // 保存这些 candidate，candidate 信息主要包括 IP 和端口号，以及所采用的协议类型等
        if (msg.type === 'candidate') {
            const candidate = new RTCIceCandidate(JSON.parse(event.data.candidate));
            this.peerConnection.addIceCandidate(candidate);
        }
        if (msg.type === 'hangup') {
            this.hangup();
        }
    }
    // 发送消息_方法
    postMessage(msg: any) {
        this.curBroadcas.postMessage(msg);
        
    }
    // 收到来电后创建全新的
    handleCreateNewPerrc() {
        // 停止媒体
        this.handleStreamStop();
        // 最好每一次通话都单独创建一个RTCPeerConnection对象,防止复用导致ICE候选的收集受到之前连接的影响,导致画面延迟加载,或其它异常问题无法排查处理;
        this.peerConnection = new RTCPeerConnection();
        this.peerConnection.ontrack = (event) => this.handleOnTrack(event);
        this.peerConnection.oniceconnectionstatechange  = (event) => this.handleOniceconnectionstatechange();
    }
    // 处理页面刷新和关闭方法
    handlePageRefreshClose() {
        window.addEventListener('beforeunload', () => {
            this.postMessage({data: {event: 'otherPageRefresh', eventName: '其它页面刷新了'}});
        });
        window.addEventListener('beforeunload', () => {
            this.postMessage({data: {event: 'otherPageClose', eventName: '其它页面关闭了'}});
        });
    }
    // 处理媒体停止
    handleStreamStop() {
        if (!this.peerConnection) { return; }
        let localStream = this.peerConnection.getSenders();
        localStream.forEach((item: any) => {
            item.track.stop();
        })
    }
    // 卸载
    unmount() {
        if (this.peerConnection) {
            let localStream = this.peerConnection.getSenders();
            localStream.forEach((item: any) => {
                item.track.stop();
            })
            this.peerConnection.close();
            this.peerConnection.onicecandidate = null;
            this.peerConnection.ontrack = null;
            this.peerConnection.oniceconnectionstatechange = null;
            this.peerConnection = null;
        }
        if (this.curBroadcas) {
            this.curBroadcas.onmessage = null;
            this.curBroadcas = null;
        }
    }
    // 挂断
    hangup() {
        if (!this.peerConnection) {
            return;
        }
        if (this.peerConnection.signalingState === 'closed') {
            return;
        };
        this.postMessage({type: 'hangup'});
        // 停止媒体流
        let localStream = this.peerConnection.getSenders();
        localStream.forEach((item: any) => {
            item.track.stop();
        });
        // 关闭peerConection
        this.peerConnection.close();
    }
    // ICE连接状态回调
    handleOniceconnectionstatechange() {
        // 1.检查网络配置
        if (this.peerConnection.iceConnectionState === 'checking') {
            // 发送订阅消息_给外部
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'checking', eventName: '检查网络配置'});
            // 2.ICE候选者被交换并成功建立了数据传输通道
        } else if (this.peerConnection.iceConnectionState === 'connected') {
            // 发送订阅消息_给外部
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'connected', eventName: 'ICE候选者被交换并成功建立了数据传输通道'});
            // 3.当连接被关闭或由于某种原因（如网络故障、对端关闭连接等）中断时
        } else if (this.peerConnection.iceConnectionState === 'disconnected') {
            this.hangup();
            this.onSubscriptionMsg({event: 'iceConnectionState', code: 'connected', eventName: 'ICE接被关闭或由于某种原因断开'});
        };
    }
    // 发送订阅消息给外部
    onSubscriptionMsg(msg: {}) {
    }
    // 接听
    answer(msg: any, stream?: MediaStream) {
        return new Promise((resolve, reject) => {
            this.handleStreamAddPeerConnection(stream);
            this.handleSedRemoteSDP(msg).then(() => {
                this.handleCreateAnswer().then((offer) => {
                    this.handleLocalDes(offer).then(() => {
                        this.handleSendAnswerRemoteMsg();
                        resolve({code: 1, message: '已发送接听消息给发送端,等待ice确认'});
                    }).catch(() => {
                        reject({code: 0, message: '本地sdp存储失败'});
                    });
                }).catch(() => {
                    reject({code: 0, message: '创建接听offer失败'});
                });
            }).catch(() => {
                reject({code: 0, message: '远端sdp存储失败'});
            })
        });
    }
    // 1.获取本地媒体流
    getUserMediaToStream(audio: true, video: true) {
        return navigator.mediaDevices.getUserMedia({audio, video});
    }
    // 2.把媒体流轨道添加到 this.peerConnection 中
    handleStreamAddPeerConnection(stream?: MediaStream) {
        if (!stream) {
            stream = new MediaStream();
        }
        const tmpStream = new MediaStream();
        const audioTracks = stream.getAudioTracks();
        const videoTracks = stream.getVideoTracks();
        if (audioTracks.length) {
            tmpStream.addTrack(audioTracks[0]);
            this.peerConnection.addTrack(tmpStream.getAudioTracks()[0], tmpStream);
        }
        if (videoTracks.length) {
            tmpStream.addTrack(videoTracks[0]);
            this.peerConnection.addTrack(tmpStream.getVideoTracks()[0], tmpStream);
        }
    }
    // 3.收到远端邀请_存远端SDP
    handleSedRemoteSDP(msg: any) {
        const answerData = msg;
        const answer = new RTCSessionDescription(answerData);
        return this.peerConnection.setRemoteDescription(answer);
    }
    // 4.接听
    handleCreateAnswer() {
        return this.peerConnection.createAnswer();
    }
    // 5.设置本地SDP描述
    handleLocalDes(offer) {
        return this.peerConnection.setLocalDescription(offer);
    }
    // 6.发送接听消息给远端
    handleSendAnswerRemoteMsg() {
        const answerData = {
            type: this.peerConnection.localDescription.type,
            sdp: this.peerConnection.localDescription.sdp
        };
        // 使用某种方式将answer传递回窗口A
        this.curBroadcas.postMessage(answerData);
    }
    // 7.监听轨道赋值给video标签onTrack
    handleOnTrack(event: any) {
        let remoteStream = event.streams[0];
        // 发送订阅消息_给外部
        this.onSubscriptionMsg({event: 'remoteStreams', eventName: '远端视频准备好了', remoteStream})
    }
}
导出方法
typescript 复制代码 
// 创建广播通道_建立两个窗口的广播通道,方便互发消息
function CreateBroadcastChannel(channelName: string) {
    return new BroadcastChannel(channelName);
};
export {CallWindowWebRtc, AnswerWindowWebRtc};
vue3主窗口使用
typescript 复制代码 
<template>
    <div class="root">
        <h1>主窗口</h1>
        <div class="loca_right_parent_wrap">
            <div class="loca_video_wrap">
                <div>
                    <button @click="methods.handleVideoToTracks()">发送本地视频给兄弟窗口</button>
                </div>
                <video id="locaVideo" autoplay controls src="./one.mp4" loop width="640" height="480" muted></video>
            </div>
            <div class="remote_video_wrap">
                <div class="tip_text">兄弟窗口视频预览 <button @click="methods.handleHangUp()">挂断</button> </div>
                <video id="remoteVideo" autoplay controls width="640" height="480"></video>
            </div>
        </div>
    </div>
</template>


<script lang="ts">
import {onMounted, onBeforeUnmount} from 'vue';
import {CallWindowWebRtc} from '@/Util/MultiWindowSharingStream';
let curWebRtc: any = null;
export default {
    setup() {
        const methods = {
            handleVideoToTracks() {
                if (curWebRtc) {
                    curWebRtc.unmount && curWebRtc.unmount();
                }
                curWebRtc = new CallWindowWebRtc({});
                // 获取轨道
                const myVideo = document.getElementById('locaVideo');
                const myVideoStream = (myVideo as any).captureStream(30);
                // 呼叫
                curWebRtc.call(myVideoStream);
                
                // 拦截订阅消息
                curWebRtc.onSubscriptionMsg = (msg) => {
                    if (msg.event && msg.event === 'remoteStreams') {
                        const {remoteStream} = msg;
                        const remoteRef = document.getElementById('remoteVideo');
                        (remoteRef as HTMLVideoElement).srcObject = remoteStream;
                        // (remoteRef as HTMLVideoElement).play();
                    }
                }

            },
            handleHangUp() {
                if (curWebRtc) {
                    curWebRtc.hangup && curWebRtc.hangup();
                }
            },
            // 处理组件卸载
            handleUnmount(){
                if (curWebRtc) {
                    curWebRtc.unmount && curWebRtc.unmount();
                }
            }
        }
        onMounted(() => {
           
        });
        onBeforeUnmount(() => {
            methods.handleUnmount();
            
        })
        return {
            methods,
        }
    }
}
</script>

<style lang="scss" scoped>
.root{
    .loca_right_parent_wrap{
        display: flex;
    }
    .loca_video_wrap{
        box-sizing: border-box;
        padding: 0 5px;
        video{
            background: #000;
        }
    }
    .remote_video_wrap{
        box-sizing: border-box;
        padding: 0 5px;
        .tip_text{
            height: 28px;
        }
        video{
            background: #000;
        }
    }
}
</style>
vue3兄弟窗口使用
typescript 复制代码 
<template>
    <div class="root">
        <h1>兄弟窗口</h1>
        <div class="loca_right_parent_wrap">
            <div class="loca_video_wrap">
                <div class="tip_text">本地视频预览</div>
                <video id="locaVideo" autoplay controls src="./two.mp4" loop width="640" height="480"></video>
            </div>
            <div class="remote_video_wrap">
                <div class="tip_text">主窗口视频预览 <button @click="methods.handleHangUp()">挂断</button></div>
                <video id="remoteVideo" autoplay controls width="640" height="480"></video>
            </div>
        </div>
    </div>
</template>


<script lang="ts">
import {onMounted, onBeforeUnmount} from 'vue';
import {AnswerWindowWebRtc} from '@/Util/MultiWindowSharingStream';
let curWebRtc: any = null;
export default {
    setup() {
        const methods = {
            handleVideoToTracks() {
            },
            handleInitAnswerOne() {
                if (curWebRtc) {
                    curWebRtc.close && curWebRtc.close();
                    curWebRtc = null;
                }
                curWebRtc = new AnswerWindowWebRtc({});
                const remoteRef = document.getElementById('remoteVideo');
                const myVideo = document.getElementById('locaVideo');
                // 拦截订阅消息
                curWebRtc.onSubscriptionMsg = (msg) => {
                    // 收到远端媒体流
                    if (msg.event && msg.event === 'remoteStreams') {
                        const {remoteStream} = msg;
                        const remoteRef = document.getElementById('remoteVideo');
                        (remoteRef as HTMLVideoElement).srcObject = remoteStream;
                        // (remoteRef as HTMLVideoElement).play();
                    }
                    // 收到新的来电
                    if (msg.event && msg.event === 'incomingCall') {
                        (remoteRef as HTMLVideoElement).srcObject = null;
                        // 获取轨道
                        const locaVideoStream = (myVideo as any).captureStream(30);
                        const {offer} = msg;
                        curWebRtc.answer(offer, locaVideoStream);
                    }
                }

            },
            handleHangUp() {
                if (curWebRtc) {
                    curWebRtc.hangup && curWebRtc.hangup();
                }
            },
            handleUnmount() {
                if (curWebRtc) {
                    curWebRtc.unmount && curWebRtc.unmount();
                    curWebRtc = null;
                }
            }
        }
        onMounted(() => {
            methods.handleInitAnswerOne();
        });
        onBeforeUnmount(() => {
            methods.handleUnmount();
        });
        return {
            methods
        }
    }
}
</script>

<style lang="scss" scoped>
.root{
    .loca_right_parent_wrap{
        display: flex;
    }
    .loca_video_wrap{
        box-sizing: border-box;
        padding: 0 5px;
        .tip_text{
            height: 28px;
        }
        video{
            background: #000;
        }
    }
    .remote_video_wrap{
        box-sizing: border-box;
        padding: 0 5px;
        .tip_text{
            height: 28px;
        }
        video{
            background: #000;
        }
    }
}
</style>
相关推荐
EasyCVR25 分钟前
EHOME视频平台EasyCVR视频融合平台使用OBS进行RTMP推流，WebRTC播放出现抖动、卡顿如何解决？
人工智能·算法·ffmpeg·音视频·webrtc·监控视频接入
安步当歌4 小时前
【WebRTC】视频编码链路中各个类的简单分析——VideoStreamEncoder
音视频·webrtc·视频编解码·video-codec
安步当歌18 小时前
【WebRTC】视频采集模块中各个类的简单分析
音视频·webrtc·视频编解码·video-codec
wyw00001 天前
解决SRS推送webrtc流卡顿问题
webrtc·srs
安步当歌4 天前
【WebRTC】WebRTC的简单使用
音视频·webrtc
西部秋虫4 天前
Windows下FFmpeg集成metaRTC实现webrtc推拉流的例子
ffmpeg·webrtc
johnny2336 天前
《Web性能权威指南》-WebRTC-读书笔记
webrtc
xiejiashu7 天前
EasyPlayer.js网页播放器,支持FLV、HLS、WebSocket、WebRTC、H.264/H.265、MP4、ts各种音视频流播放
websocket·webrtc·h.264·flv播放器·hls播放器
Dreamt灬7 天前
webrtc agc2实现原理
webrtc
程序员阿灿8 天前
WebRTC学习六：MediaStream 常用API介绍
音视频·webrtc
热门推荐
01【HarmonyOS】HUAWEI DevEco Studio 下载地址汇总02(欧拉)openEuler系统添加网卡文件配置流程、(欧拉)openEuler系统手动配置ipv6地址流程、(欧拉)openEuler系统网络管理说明03组基轨迹建模 GBTM的介绍与实现（Stata 或 R）04【AIGC】重塑未来的科技巨轮05全面解析：构建基于深度学习的安全帽检测系统（UI界面+YOLO代码+数据集）06【经验分享】Ubuntu22.04安装微信(linux官方版)07基于YOLOv10深度学习的CT扫描图像肾结石智能检测系统【python源码+Pyqt5界面+数据集+训练代码】深度学习实战、目标检测08Ubuntu 20.04使用Livox mid 360 测试 FAST_LIO09RAG 实践- Ollama+RagFlow 部署本地知识库10【TC3xx芯片】TC3xx芯片电源管理系统PMS详解