介绍FPGA通过MIPI CSI-2发送实时图像到RK3588，并HDMI显示。
FPGA本地产生动态图像模板，通过MIPI CSI-2接口发送到RK3588 MIPI CSI接口。RK3588注册成相机后，调用接口并在HDMI显示器上显示。
1、RK3588驱动调试
查看Media controller信息
Media controller API version 6.1.84
Media device information
driver rkcif
model rkcif-mipi-lvds
serial
bus info platform:rkcif-mipi-lvds
hw revision 0x0
driver version 6.1.84
Device topology
- entity 1: stream_cif_mipi_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video0
pad0: Sink
<- "rockchip-mipi-csi2":1 [ENABLED]
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 5: stream_cif_mipi_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video1
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 [ENABLED]
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 9: stream_cif_mipi_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video2
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 [ENABLED]
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 13: stream_cif_mipi_id3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video3
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 [ENABLED]
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 17: rkcif_scale_ch0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video4
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 [ENABLED]
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 21: rkcif_scale_ch1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video5
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 [ENABLED]
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 25: rkcif_scale_ch2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video6
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 [ENABLED]
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 29: rkcif_scale_ch3 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video7
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 [ENABLED]
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 33: rkcif_tools_id0 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video8
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 [ENABLED]
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 []
- entity 37: rkcif_tools_id1 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video9
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 [ENABLED]
<- "rockchip-mipi-csi2":11 []
- entity 41: rkcif_tools_id2 (1 pad, 11 links)
type Node subtype V4L flags 0
device node name /dev/video10
pad0: Sink
<- "rockchip-mipi-csi2":1 []
<- "rockchip-mipi-csi2":2 []
<- "rockchip-mipi-csi2":3 []
<- "rockchip-mipi-csi2":4 []
<- "rockchip-mipi-csi2":5 []
<- "rockchip-mipi-csi2":6 []
<- "rockchip-mipi-csi2":7 []
<- "rockchip-mipi-csi2":8 []
<- "rockchip-mipi-csi2":9 []
<- "rockchip-mipi-csi2":10 []
<- "rockchip-mipi-csi2":11 [ENABLED]
- entity 45: rockchip-mipi-csi2 (12 pads, 122 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev0
pad0: Sink
[fmt:SBGGR8_1X8/2304x1296 field:none
crop.bounds:(0,0)/2304x1296
crop:(0,0)/2304x1296]
<- "rockchip-csi2-dphy0":1 [ENABLED]
pad1: Source
-> "stream_cif_mipi_id0":0 [ENABLED]
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad2: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 [ENABLED]
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad3: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 [ENABLED]
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad4: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 [ENABLED]
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad5: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 [ENABLED]
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad6: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 [ENABLED]
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad7: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 [ENABLED]
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad8: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 [ENABLED]
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad9: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 [ENABLED]
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 []
pad10: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 [ENABLED]
-> "rkcif_tools_id2":0 []
pad11: Source
-> "stream_cif_mipi_id0":0 []
-> "stream_cif_mipi_id1":0 []
-> "stream_cif_mipi_id2":0 []
-> "stream_cif_mipi_id3":0 []
-> "rkcif_scale_ch0":0 []
-> "rkcif_scale_ch1":0 []
-> "rkcif_scale_ch2":0 []
-> "rkcif_scale_ch3":0 []
-> "rkcif_tools_id0":0 []
-> "rkcif_tools_id1":0 []
-> "rkcif_tools_id2":0 [ENABLED]
- entity 58: rockchip-csi2-dphy0 (2 pads, 2 links)
type V4L2 subdev subtype Unknown flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none
crop.bounds:(0,0)/2304x1296]
<- "m00_b_imx415 7-001a":0 [ENABLED]
pad1: Source
-> "rockchip-mipi-csi2":0 [ENABLED]
- entity 63: m00_b_imx415 7-001a (1 pad, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Source
[fmt:SBGGR8_1X8/2304x1296@10000/300000 field:none
crop.bounds:(0,0)/2304x1296]
-> "rockchip-csi2-dphy0":0 [ENABLED]
2、RK3588手动抓取FPGA图像
v4l2-ctl -d /dev/video0 --set-fmt-video=width=2304,height=1296,pixelformat=BA81 \--stream-mmap=3 \--stream-skip=3 \--stream-to=/tmp/cif888.out \--stream-count=1 \--stream-poll
[ 14.624791] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream
[ 14.624835] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream on, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0
[ 14.624847] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream ON
[ 14.624856] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_start
[ 14.624882] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_update_sensor_info
[ 14.624904] rockchip-mipi-csi2: csi2->bus.num_data_lanes = 4
[ 14.694384] rkcif-mipi-lvds: ERROR: csi size err, intstat:0x1000001, lastline:0!!
[ 14.694434] mipi0-csi2 ERR1:0x10 (fs/fe mis,vc: 0)
<<<<[ 15.185394] rockchip-mipi-csi2 fdd10000.mipi0-csi2: hzpd csi2_s_stream
[ 15.185458] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream off, src_sd: 00000000d5480faf, sd_name:rockchip-csi2-dphy0
[ 15.185472] rockchip-mipi-csi2 fdd10000.mipi0-csi2: stream OFF
3、查看抓取的图像文件
tmp# ls -l
total 3624
-rw-r--r-- 1 root root 2985984 Mar 2 07:47 cif888.out
-rw-r--r-- 1 root root 695958 Mar 2 07:47 iq_dump.json
drwx------ 2 root root 4096 Mar 2 07:47 snap-private-tmp
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-ModemManager.service-OohSfg
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-logind.service-742rHg
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-resolved.service-L1NO8e
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-systemd-timesyncd.service-815p5g
drwx------ 3 root root 4096 Mar 2 07:47 systemd-private-54d04dbd901a4c6d9bf51f22a372453e-upower.service-24inCg
drwx------ 2 firefly firefly 4096 Mar 2 07:47 tracker-extract-files.1000
srwxr-xr-x 1 root root 0 Mar 2 07:47 UNIX.domain
4、运行GST测试，HDMI显示测试
gst-launch-1.0 videotestsrc ! videoconvert ! autovideosink
Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Pipeline is PREROLLED ...
Prerolled, waiting for async message to finish...
Setting pipeline to PLAYING ...
Redistribute latency...
New clock: GstSystemClock
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:07.325302254
Setting pipeline to NULL ...
Freeing pipeline ...
测试效果如图。
5、运行GST，显示FPGA实时发送图像
RK3588和FPGA启动运行正常后，在RK3588输入命令。
gst-launch-1.0 v4l2src device=/dev/video-camera0 ! video/x-raw,for
root@rk3588:/#
root@rk3588:/#
rmat=RG10,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)RG10, width=(int)2304, height=(int)1296, framerate=(fraction)30/1
root@rk3588:/#
root@rk3588:/#
rmat=BA81,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
WARNING: erroneous pipeline: could not link v4l2src0 to waylandsink0, neither element can handle caps video/x-raw, format=(string)BA81, width=(int)2304, height=(int)1296, framerate=(fraction)30/1
root@rk3588:/#
root@rk3588:/#
root@rk3588:/#
rmat=UYVY,width=2304,height=1296, framerate=30/1 ! waylandsink0 ! video/x-raw,for
Setting pipeline to PAUSED ...
Using mplane plugin for capture
Pipeline is live and does not need PREROLL ...
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
../libs/gst/base/gstbasesrc.c(3132): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000817834
Setting pipeline to NULL ...
Freeing pipeline ...
实时显示FPGA发送的动态图像，效果如下。
6、欢迎讨论交流，方案定制设计。