Android Exoplayer多路不同时长音视频混合播放

在上一篇Android Exoplayer 实现多个音视频文件混合播放以及音轨切换中我们提到一个问题，如果视频和音频时长不一致，特别是想混合多个音频和多个视频时就会出问题，无法播放。报错如下：

复制代码

E/ExoPlayerImplInternal(11191): Playback error
E/ExoPlayerImplInternal(11191):   com.google.android.exoplayer2.ExoPlaybackException: Source error
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleIoException(ExoPlayerImplInternal.java:684)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.ExoPlayerImplInternal.handleMessage(ExoPlayerImplInternal.java:660)
E/ExoPlayerImplInternal(11191):       at android.os.Handler.dispatchMessage(Handler.java:98)
E/ExoPlayerImplInternal(11191):       at android.os.Looper.loop(Looper.java:136)
E/ExoPlayerImplInternal(11191):       at android.os.HandlerThread.run(HandlerThread.java:61)
E/ExoPlayerImplInternal(11191):   Caused by: com.google.android.exoplayer2.source.MergingMediaSource$IllegalMergeException
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.MergingMediaSource.onChildSourceInfoRefreshed(MergingMediaSource.java:252)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.MergingMediaSource.onChildSourceInfoRefreshed(MergingMediaSource.java:52)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.CompositeMediaSource.lambda$prepareChildSource$0$com-google-android-exoplayer2-source-CompositeMediaSource(CompositeMediaSource.java:120)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.CompositeMediaSource$$ExternalSyntheticLambda0.onSourceInfoRefreshed(D8$$SyntheticClass:0)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.BaseMediaSource.refreshSourceInfo(BaseMediaSource.java:94)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.updateTimelineAndScheduleOnCompletionActions(ConcatenatingMediaSource.java:746)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.handleMessage(ConcatenatingMediaSource.java:716)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource.$r8$lambda$xvlxaabNVihM68DRWdn_WPenrXk(ConcatenatingMediaSource.java)
E/ExoPlayerImplInternal(11191):       at com.google.android.exoplayer2.source.ConcatenatingMediaSource$$ExternalSyntheticLambda0.handleMessage(D8$$SyntheticClass:0)
E/ExoPlayerImplInternal(11191):       ... 3 more

这个主要是播放时长不一致，无法同步时序导致。

使用场景：比如K歌应用中，没有原版MV，只有音频文件，想给音频配一个背景视频或多个混合视频当MV，但视频均是风景短片，时长与音频不一致。当用Exoplayer进行混合播放时，我们希望以音频时长为准，在音频播放完成前，视频循环播放。

一直没有找到很好的方法解决，最后采取了一个笨办法，启用两个播放器，一个专门播放视频，一个专门播放音频，这样视频任意混合或循环播放都与音频互不干扰，就可用规避时序错乱问题。

以下为实现样例：

复制代码

 private ExoPlayer mExoPlayer,mExoPlayer2;

  //音频播放器
 mExoPlayer = new ExoPlayer.Builder(context, renderersFactory)
                .setTrackSelector(trackSelector)
                .build();
 // 检查音频配置
        AudioAttributes audioAttributes = new AudioAttributes.Builder()
                .setUsage(C.USAGE_MEDIA)
                .setContentType(C.AUDIO_CONTENT_TYPE_MUSIC)
                .build();
 mExoPlayer.setAudioAttributes(audioAttributes, true);

    //视频播放器
        mExoPlayer2 = new ExoPlayer.Builder(context, new DefaultMediaSourceFactory(context))
                .setVideoScalingMode(MediaCodec.VIDEO_SCALING_MODE_SCALE_TO_FIT)
                .build();


 DefaultMediaSourceFactory mediaSourceFactory = new DefaultMediaSourceFactory(dataSourceFactory);
            // 创建两个视频的 MediaSource
            MediaSource video1Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/01.mp4"));
            MediaSource video2Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/02.mp4"));
            ConcatenatingMediaSource concatenatingMediaSource = new ConcatenatingMediaSource(video1Source,video2Source);
            LoopingMediaSource loopingMediaSource = new LoopingMediaSource(concatenatingMediaSource);
            // 合并两个音频源
            MediaSource audio1Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/audio/ori.mp2"));
            MediaSource audio2Source = mediaSourceFactory.createMediaSource(MediaItem.fromUri("asset://android_asset/audio/acc.mp2"));
            MergingMediaSource audioMerged = new MergingMediaSource(audio1Source, audio2Source);
            mExoPlayer2.setMediaSource(loopingMediaSource);
           // mExoPlayer2.setRepeatMode(Player.REPEAT_MODE_ONE);
            // 添加到 ExoPlayer
            mExoPlayer.setMediaSource(audioMerged);

需要注意的是两个播放器要保持状态同步，以播放进度音频播放器为准。

复制代码

   @Override
    public void prepareAsync() throws IllegalStateException {
       
        mExoPlayer.prepare();
        mExoPlayer2.prepare();
    }

    @Override
    public void start() throws IllegalStateException {
    
        mExoPlayer.setPlayWhenReady(true);
        mExoPlayer2.setPlayWhenReady(true);
        //  getCurrentPostion();
    }

    @Override
    public void stop() throws IllegalStateException {
        mExoPlayer.stop();
        mExoPlayer2.stop();
    }

    @Override
    public void pause() throws IllegalStateException {
   
        mExoPlayer.setPlayWhenReady(false);
        mExoPlayer2.setPlayWhenReady(false);
    }

    @Override
    public void setSpeed(float speed) {
        PlaybackParameters parameters = new PlaybackParameters(speed);
        mExoPlayer.setPlaybackParameters(parameters);
    }

    @Override
    public long getCurrentPosition() {
        if (mExoPlayer == null)
            return 0;
        return mExoPlayer.getCurrentPosition();
    }

还有就是切换音轨的时候需要注意，由于音视频分开处理了，切换音轨的时候只处理音频播放器即可，切换分辨率的时候只处理视频播放器即可，这时媒体轨道数会比音视频混合一起的情况要少一些，因为只有视频或只有音频轨道，切换时轨道索引值参数肯定要小些了。

比如上面样例音频播放器只有两个音频轨道，所以切换音轨时，索引只有0或1.

复制代码

//原唱
TrackGroup selectedGroup = currentTrackGroups.get(0);
//伴奏
//TrackGroup selectedGroup = currentTrackGroups.get(1);

// 应用新音轨
  mExoPlayer.setTrackSelectionParameters(mExoPlayer.getTrackSelectionParameters()
             .buildUpon()
              .setOverrideForType(new TrackSelectionOverride(selectedGroup, 0))  //需要切换到的音轨索引
               .build());

这样就可以迂回解决多路不同时长音视频混合流播放问题。大佬们有其他更好解决办法的话欢饮留言交流。