介绍FPGA通过MIPI CSI-2发送实时图像到RK3588，并HDMI显示。
FPGA本地产生动态图像模板，通过MIPI CSI-2接口发送到RK3588 MIPI CSI接口。RK3588注册成相机后，调用接口并在HDMI显示器上显示。
1、RK3588驱动调试
查看Media controller信息
Media controller API version 6.1.84
Media device information
driver rkcif
model rkcif-mipi-lvds
serial
bus info platform:rkcif-mipi-lvds
hw revision 0x0
driver version 6.1.84
Device topology
- entity 1: stream_cif_mipi_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "rockchip-mipi-csi2":1 [ENABLED]
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 5: stream_cif_mipi_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 [ENABLED]
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 9: stream_cif_mipi_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 [ENABLED]
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 13: stream_cif_mipi_id3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 [ENABLED]
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 17: rkcif_scale_ch0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video4
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 [ENABLED]
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 21: rkcif_scale_ch1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video5
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 [ENABLED]
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 25: rkcif_scale_ch2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video6
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 [ENABLED]
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 29: rkcif_scale_ch3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video7
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 [ENABLED]
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 33: rkcif_tools_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video8
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 [ENABLED]
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 37: rkcif_tools_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video9
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 [ENABLED]
<- "rockchip-mipi-csi2":11 []
- entity 41: rkcif_tools_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video10
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 [ENABLED]
- entity 45: rockchip-mipi-csi2 (12 pads, 122 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
fmt:SBGGR8_1X8/2304x1296 field:none crop.bounds:(0,0)/2304x1296 crop:(0,0)/2304x1296
<- "rockchip-csi2-dphy0":1 [ENABLED]
pad1: Source
-> "stream_cif_mipi_id0":0 [ENABLED]
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad2: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 [ENABLED]
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad3: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 [ENABLED]
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad4: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 [ENABLED]
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad5: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 [ENABLED]
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad6: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 [ENABLED]
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad7: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 [ENABLED]
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad8: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 [ENABLED]
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad9: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 [ENABLED]
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad10: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 [ENABLED]
-> "rkcif_tools_id2":0 []
pad11: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 [ENABLED]
- entity 58: rockchip-csi2-dphy0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none crop.bounds:(0,0)/2304x1296
<- "m00_b_imx415 7-001a":0 [ENABLED]
pad1: Source
-> "rockchip-mipi-csi2":0 [ENABLED]
- entity 63: m00_b_imx415 7-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none crop.bounds:(0,0)/2304x1296
-> "rockchip-csi2-dphy0":0 [ENABLED]
2、RK3588手动抓取FPGA图像
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2304,height=1296,pixelformat=BA81 \--stream-mmap=3 \--stream-skip=3 \--stream-to=/tmp/cif888.out \--stream-count=1 \--stream-poll
14.624791\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream \[ 14.624835\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0 \[ 14.624847\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON \[ 14.624856\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_start \[ 14.624882\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_update_sensor_info \[ 14.624904\] rockchip-mipi-csi2: csi2-\>bus.num_data_lanes = 4 \[ 14.694384\] rkcif-mipi-lvds: ERROR: csi size err, intstat:0x1000001, lastline:0!! \[ 14.694434\] mipi0-csi2 ERR1:0x10 (fs/fe mis,vc: 0) \<\<\<\<\[ 15.185394\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream \[ 15.185458\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0 \[ 15.185472\] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF 3、查看抓取的图像文件 tmp# ls -l total 3624 -rw-r--r-- 1 root root 2985984 Mar 2 07:47 cif888.out -rw-r--r-- 1 root root 695958 Mar 2 07:47 iq_dump.json drwx------ 2 root root 4096 Mar 2 07:47 snap-private-tmp drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-ModemManager.service-OohSfg drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-logind.service-742rHg drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-resolved.service-L1NO8e drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-timesyncd.service-815p5g drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-upower.service-24inCg drwx------ 2 firefly firefly 4096 Mar 2 07:47 tracker-extract-files.1000 srwxr-xr-x 1 root root 0 Mar 2 07:47 UNIX.domain 4、运行GST测试，HDMI显示测试 gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink Setting pipeline to PAUSED ... Pipeline is PREROLLING ... Pipeline is PREROLLED ... Prerolled, waiting for async message to finish... Setting pipeline to PLAYING ... Redistribute latency... New clock: GstSystemClock \^Chandling interrupt. Interrupt: Stopping pipeline ... Execution ended after 0:00:07.325302254 Setting pipeline to NULL ... Freeing pipeline ... 测试效果如图。  5、运行GST，显示FPGA实时发送图像  RK3588和FPGA启动运行正常后，在RK3588输入命令。 gst-launch-1.0 v4l2src device=/dev/video-camera0 ! video/x-raw,for root@rk3588:/# root@rk3588:/# rmat=RG10,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)RG10, width=(int)2304, height=(int)1296, framerate=(fraction)30/1 root@rk3588:/# root@rk3588:/# rmat=BA81,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)BA81, width=(int)2304, height=(int)1296, framerate=(fraction)30/1 root@rk3588:/# root@rk3588:/# root@rk3588:/# rmat=UYVY,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for Setting pipeline to PAUSED ... Using mplane plugin for capture Pipeline is live and does not need PREROLL ... Pipeline is PREROLLED ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error. Additional debug info: ../libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming stopped, reason not-negotiated (-4) Execution ended after 0:00:00.000817834 Setting pipeline to NULL ... Freeing pipeline ... 实时显示FPGA发送的动态图像，效果如下。    6、欢迎讨论交流，方案定制设计。