介绍FPGA通过MIPI CSI-2发送实时图像到RK3588，并HDMI显示。
FPGA本地产生动态图像模板，通过MIPI CSI-2接口发送到RK3588 MIPI CSI接口。RK3588注册成相机后，调用接口并在HDMI显示器上显示。
1、RK3588驱动调试
查看Media controller信息
Media controller API version 6.1.84
Media device information
driver rkcif
model rkcif-mipi-lvds
serial
bus info platform:rkcif-mipi-lvds
hw revision 0x0
driver version 6.1.84
Device topology
- entity 1: stream_cif_mipi_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "rockchip-mipi-csi2":1 ENABLED
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 5: stream_cif_mipi_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 ENABLED
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 9: stream_cif_mipi_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 ENABLED
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 13: stream_cif_mipi_id3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 ENABLED
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 17: rkcif_scale_ch0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video4
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 ENABLED
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 21: rkcif_scale_ch1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video5
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 ENABLED
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 25: rkcif_scale_ch2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video6
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 ENABLED
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 29: rkcif_scale_ch3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video7
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 ENABLED
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 33: rkcif_tools_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video8
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 ENABLED
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 \[\]
- entity 37: rkcif_tools_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video9
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 ENABLED
<- "rockchip-mipi-csi2":11 \[\]
- entity 41: rkcif_tools_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video10
pad0: Sink
<- "rockchip-mipi-csi2":1 \[\]
<- "rockchip-mipi-csi2":2 \[\]
<- "rockchip-mipi-csi2":3 \[\]
<- "rockchip-mipi-csi2":4 \[\]
<- "rockchip-mipi-csi2":5 \[\]
<- "rockchip-mipi-csi2":6 \[\]
<- "rockchip-mipi-csi2":7 \[\]
<- "rockchip-mipi-csi2":8 \[\]
<- "rockchip-mipi-csi2":9 \[\]
<- "rockchip-mipi-csi2":10 \[\]
<- "rockchip-mipi-csi2":11 ENABLED
- entity 45: rockchip-mipi-csi2 (12 pads, 122 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SBGGR8_1X8/2304x1296 field:none
crop.bounds:(0,0)/2304x1296
crop:(0,0)/2304x1296]
<- "rockchip-csi2-dphy0":1 ENABLED
pad1: Source
-> "stream_cif_mipi_id0":0 ENABLED
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad2: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 ENABLED
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad3: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 ENABLED
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad4: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 ENABLED
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad5: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 ENABLED
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad6: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 ENABLED
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad7: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 ENABLED
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad8: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 ENABLED
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad9: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 ENABLED
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 \[\]
pad10: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 ENABLED
-> "rkcif_tools_id2":0 \[\]
pad11: Source
-> "stream_cif_mipi_id0":0 \[\]
-> "stream_cif_mipi_id1":0 \[\]
-> "stream_cif_mipi_id2":0 \[\]
-> "stream_cif_mipi_id3":0 \[\]
-> "rkcif_scale_ch0":0 \[\]
-> "rkcif_scale_ch1":0 \[\]
-> "rkcif_scale_ch2":0 \[\]
-> "rkcif_scale_ch3":0 \[\]
-> "rkcif_tools_id0":0 \[\]
-> "rkcif_tools_id1":0 \[\]
-> "rkcif_tools_id2":0 ENABLED
- entity 58: rockchip-csi2-dphy0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none
crop.bounds:(0,0)/2304x1296]
<- "m00_b_imx415 7-001a":0 ENABLED
pad1: Source
-> "rockchip-mipi-csi2":0 ENABLED
- entity 63: m00_b_imx415 7-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none
crop.bounds:(0,0)/2304x1296]
-> "rockchip-csi2-dphy0":0 ENABLED
2、RK3588手动抓取FPGA图像
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2304,height=1296,pixelformat=BA81 \--stream-mmap=3 \--stream-skip=3 \--stream-to=/tmp/cif888.out \--stream-count=1 \--stream-poll
14.624791 rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream
14.624835 rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0
14.624847 rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON
14.624856 rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_start
14.624882 rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_update_sensor_info
14.624904 rockchip-mipi-csi2: csi2->bus.num_data_lanes = 4
14.694384 rkcif-mipi-lvds: ERROR: csi size err, intstat:0x1000001, lastline:0!!
14.694434 mipi0-csi2 ERR1:0x10 (fs/fe mis,vc: 0)
<<<< 15.185394 rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream
15.185458 rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0
15.185472 rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF
3、查看抓取的图像文件
tmp# ls -l
total 3624
-rw-r--r-- 1 root root 2985984 Mar 2 07:47 cif888.out
-rw-r--r-- 1 root root 695958 Mar 2 07:47 iq_dump.json
drwx------ 2 root root 4096 Mar 2 07:47 snap-private-tmp
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-ModemManager.service-OohSfg
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-logind.service-742rHg
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-resolved.service-L1NO8e
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-timesyncd.service-815p5g
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-upower.service-24inCg
drwx------ 2 firefly firefly 4096 Mar 2 07:47 tracker-extract-files.1000
srwxr-xr-x 1 root root 0 Mar 2 07:47 UNIX.domain
4、运行GST测试，HDMI显示测试
gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.325302254
Setting pipeline to NULL ...
Freeing pipeline ...
测试效果如图。
5、运行GST，显示FPGA实时发送图像
RK3588和FPGA启动运行正常后，在RK3588输入命令。
gst-launch-1.0 v4l2src device=/dev/video-camera0 ! video/x-raw,for
root@rk3588:/#
root@rk3588:/#
rmat=RG10,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)RG10, width=(int)2304, height=(int)1296, framerate=(fraction)30/1
root@rk3588:/#
root@rk3588:/#
rmat=BA81,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)BA81, width=(int)2304, height=(int)1296, framerate=(fraction)30/1
root@rk3588:/#
root@rk3588:/#
root@rk3588:/#
rmat=UYVY,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
Setting pipeline to PAUSED ...
Using mplane plugin for capture
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000817834
Setting pipeline to NULL ...
Freeing pipeline ...
实时显示FPGA发送的动态图像，效果如下。
6、欢迎讨论交流，方案定制设计。