Ubuntu OpenCV C++ 获取Astra Pro摄像头图像

qq_271581792025-11-01 23:47

0. 环境

  • ubuntu18台式机

  • 奥比中光 Orbbec Astra Pro 单目结构光摄像头

  • opencv 3.4.20 + OpenNI_2.3.0.66

1. Astra Pro出厂测试

1.1 获取sdk

官网资料

https://www.orbbec.com.cn/index/Download2025/info.html?cate=121\&id=1

链接到

https://github.com/orbbec/OpenNI_SDK/tree/main

sdk

复制代码 
https://dl.orbbec3d.com/dist/openni2/OpenNI_2.3.0.66.zip
https://3dclub.orbbec3d.com/t/universal-download-thread-for-astra-series-cameras/622/12

把

OpenNI_2.3.0.66.zip

拷贝到

astrapro_out_of_box\ext

1.2 环境变量

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"

export C_COMPILER="gcc"
export CXX_COMPILER="g++"
export AR="ar"

echo $WORK_DIR
echo $ARCH
echo $C_COMPILER
echo $CXX_COMPILER
echo $AR

1.3 解压并切换目录

复制代码 
cd $WORK_DIR
cd ext/
unzip -d OpenNI_2.3.0.66 OpenNI_2.3.0.66.zip
cd OpenNI_2.3.0.66/Linux/OpenNI-Linux-x64-2.3.0.66

1.4 安装脚本

复制代码 
chmod +x install.sh
sudo ./install.sh

1.5 环境变量

复制代码 
source OpenNIDevEnvironment

1.6 测试demo：Samples/SimpleViewer

复制代码 
cd $WORK_DIR
cd ext/
cd OpenNI_2.3.0.66/Linux/OpenNI-Linux-x64-2.3.0.66
cd  Samples/SimpleViewer

1.6.1 依赖

复制代码 
sudo apt-get install freeglut3-dev

1.6.2 编译

复制代码 
make -j4

1.6.3 运行测试

复制代码 
cd Bin/x64-Release/
sudo ./SimpleViewer

2. opencv与rgb图像

2.0 opencv说明

opencv，不用加openni，opencv4的官网文档说openni可以支持astra，但是测试不通过。打不开IR摄像头。Astra Pro的RGB是UVC协议，不用SDK。

2.1 准备源码

https://opencv.org/releases/

下载了

opencv-3.4.20.tar.gz

放在

~/workspace/astrapro_out_of_box/ext

解压

cd /home/xxjianvm/workspace/astrapro_out_of_box/ext

tar -zvxf opencv-3.4.20.tar.gz

2.2 安装依赖

复制代码 
sudo apt-get install -y build-essential
sudo apt-get install -y cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev

2.3 环境变量

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"

export C_COMPILER="gcc"
export CXX_COMPILER="g++"
export AR="ar"

2.4 命令行从源码编译安装

把以下内容粘贴到终端中即可。

复制代码 
cd $WORK_DIR
cd ext/
mkdir -p install && mkdir -p install/$ARCH && mkdir -p install/$ARCH/lib && mkdir -p install/$ARCH/include
rm -rf opencv-3.4.20
tar -zvxf opencv-3.4.20.tar.gz  && cd opencv-3.4.20
rm -rf build
mkdir build && cd build
cmake  -DCMAKE_C_COMPILER=$C_COMPILER -DCMAKE_CXX_COMPILER=$CXX_COMPILER \
-DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_INSTALL_PREFIX=$WORK_DIR/ext/install/$ARCH \
-DBUILD_DOCS=OFF \
-DBUILD_EXAMPLES=OFF \
-DBUILD_TESTS=OFF \
-DBUILD_PERF_TESTS=OFF \
-DSOFTFP=OFF \
-DCMAKE_C_FLAGS="-L$WORK_DIR/ext/install/$ARCH/lib" ..
make -j8
make install

2.5 demo_rgb

把demo_rgb.cpp放到~/workspace/astrapro_out_of_box/demo文件夹中。

2.5.1 demo_rgb.cpp内容

复制代码 
#include <opencv2/opencv.hpp>
#include <iostream>
using namespace cv;
using namespace std;
int main() {
   VideoCapture cap(0); // Open default camera
   if (!cap.isOpened()) {
       cout << "Failed to open camera!" << endl;
       return -1;
   }
   namedWindow("Camera", WINDOW_NORMAL);
   while (true) {
       Mat frame;
       cap.read(frame); // Capture a frame
       if (frame.empty()) {
           cout << "Failed to read frame!" << endl;
           break;
       }
       imshow("Camera", frame); // Display the frame
       if (waitKey(1) == 'q') break; // Exit on 'q'
   }
   cap.release();
   destroyAllWindows();
   return 0;
}

2.5.2 编译可执行程序

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"

export C_COMPILER="gcc"
export CXX_COMPILER="g++"
export AR="ar"

echo $WORK_DIR
echo $ARCH
echo $C_COMPILER
echo $CXX_COMPILER
echo $AR



# 编译可执行程序
$CXX_COMPILER -o demo_rgb demo/demo_rgb.cpp -Iext/install/$ARCH/include -L$WORK_DIR/ext/install/$ARCH/lib  -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_imgcodecs -lopencv_photo -lopencv_videoio -lopencv_video

2.5.3 运行

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"
export LD_LIBRARY_PATH=$WORK_DIR/ext/install/$ARCH/lib:$LD_LIBRARY_PATH
./demo_rgb

3. openni读取depth图像并转opencv的mat格式

3.1 把sdk安装到自定义目录

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"

cd $WORK_DIR
cd ext
mkdir -p install && mkdir -p install/$ARCH && mkdir -p install/$ARCH/lib && mkdir -p install/$ARCH/include
rm -rf OpenNI_2.3.0.66
unzip -d OpenNI_2.3.0.66 OpenNI_2.3.0.66.zip
cp OpenNI_2.3.0.66/Linux/OpenNI-Linux-$ARCH-2.3.0.66/Include/* install/$ARCH/include/ -r
cp OpenNI_2.3.0.66/Linux/OpenNI-Linux-$ARCH-2.3.0.66/Redist/* install/$ARCH/lib/ -r
echo "[sdk OK]"

3.2 demo_depth.cpp

复制代码 
#include <iostream>
#include <OpenNI.h>
#include <opencv2/opencv.hpp>

openni::VideoStream**		m_streams;

int main(int argc, char** argv)
{
	openni::Status rc = openni::STATUS_OK;

	openni::Device device;
	openni::VideoStream streamDepth;
	const char* deviceURI = openni::ANY_DEVICE;
	if (argc > 1) {
		deviceURI = argv[1];
	}

	rc = openni::OpenNI::initialize();

	printf("After initialization:\n%s\n", openni::OpenNI::getExtendedError());

	rc = device.open(deviceURI);
	if (rc != openni::STATUS_OK) {
		printf("SimpleViewer: Device open failed:\n%s\n", openni::OpenNI::getExtendedError());
		openni::OpenNI::shutdown();
		return 1;
	}

	rc = streamDepth.create(device, openni::SENSOR_DEPTH);
	if (rc == openni::STATUS_OK) {
		rc = streamDepth.start();
		if (rc != openni::STATUS_OK) {
			printf("SimpleViewer: Couldn't start depth stream:\n%s\n", openni::OpenNI::getExtendedError());
			streamDepth.destroy();
		}
	} else {
		printf("SimpleViewer: Couldn't find depth stream:\n%s\n", openni::OpenNI::getExtendedError());
	}

	openni::VideoMode depthVideoMode;
	int			m_width;
	int			m_height;
	if (!streamDepth.isValid()) {
		printf("SimpleViewer: No valid streams. Exiting\n");
		openni::OpenNI::shutdown();
		return 2;
	} else {
		depthVideoMode = streamDepth.getVideoMode();
		m_width = depthVideoMode.getResolutionX();
		m_height = depthVideoMode.getResolutionY();
	}


	m_streams = new openni::VideoStream*[1];
	m_streams[0] = &streamDepth;

	int iMaxDepth = streamDepth.getMaxPixelValue();
	openni::VideoFrameRef frameDepth;
	for( int i = 0; i < 100; ++ i ) {
		int changedIndex;
		openni::Status rc = openni::OpenNI::waitForAnyStream(m_streams, 1, &changedIndex);
		if (rc != openni::STATUS_OK) {
			printf("Wait failed\n");
			continue;
		}
		streamDepth.readFrame( &frameDepth );

		const cv::Mat mImageDepth( frameDepth.getHeight(), frameDepth.getWidth(), CV_16UC1, (void*)frameDepth.getData());
		cv::Mat mScaledDepth;
		mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepth );
		cv::imshow("Depth", mScaledDepth);

		if (cv::waitKey(30) == 'q') {
			break;
		}
	}

	streamDepth.destroy();
	device.close();
	openni::OpenNI::shutdown();

}

3.3 编译可执行程序

复制代码 
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"
$CXX_COMPILER -o demo_depth demo/demo_depth.cpp -Iext/install/$ARCH/include -Lext/install/$ARCH/lib  -lOpenNI2 -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_imgcodecs -lopencv_photo -lopencv_videoio -lopencv_video

3.4 运行

复制代码 
export WORK_DIR=$PWD
export ARCH="x64"
export LD_LIBRARY_PATH=$OPENNI2_REDIST:$WORK_DIR/ext/install/$ARCH/lib:$LD_LIBRARY_PATH
./demo_depth

4. 驱动

4.1 m_device.cpp

把m_device.cpp放到 ~/workspace/astrapro_out_of_box/src

复制代码 
//
// Created by qjfen on 2024/1/19.
//

#include "stdio.h"
#include <cstring>              // memset
#include <iostream>             // std::cout
#include "m_device.h"

#include <opencv2/opencv.hpp>
#include <OpenNI.h>

int handle;
cv::VideoCapture g_RgbCap;
openni::VideoStream**		m_streams;
openni::Device device;
openni::VideoStream streamDepth;
const char* deviceURI = openni::ANY_DEVICE;
 
#define SAMPLE_READ_WAIT_TIMEOUT 2000 //2000ms


m_device::m_device()
{
	//ctor
}

m_device::~m_device()
{
	//dtor
}
m_device& m_device::getInstance()
{
	static m_device instance;
	return instance;
}
int m_device::init(int handle_index)
{

	int ret;
	handle = handle_index;

	openni::Status rc = openni::STATUS_OK;
	rc = openni::OpenNI::initialize();
	printf("After initialization:\n%s\n", openni::OpenNI::getExtendedError());

	return 0;
}
int m_device::open()
{

	int ret;
	g_RgbCap = cv::VideoCapture(0);
	if (!g_RgbCap.isOpened()) {
		std::cout << "Failed to open camera!" << std::endl;
		return -1;
	}

	openni::Status rc = openni::STATUS_OK;
	rc = device.open(deviceURI);
	if (rc != openni::STATUS_OK) {
		printf("SimpleViewer: Device open failed:\n%s\n", openni::OpenNI::getExtendedError());
		openni::OpenNI::shutdown();
		return 1;
	}
	rc = streamDepth.create(device, openni::SENSOR_DEPTH);
	if (rc == openni::STATUS_OK) {
		rc = streamDepth.start();
		if (rc != openni::STATUS_OK) {
			printf("SimpleViewer: Couldn't start depth stream:\n%s\n", openni::OpenNI::getExtendedError());
			streamDepth.destroy();
		}
	} else {
		printf("SimpleViewer: Couldn't find depth stream:\n%s\n", openni::OpenNI::getExtendedError());
	}
	openni::VideoMode depthVideoMode;
	int			m_width;
	int			m_height;
	if (!streamDepth.isValid()) {
		printf("SimpleViewer: No valid streams. Exiting\n");
		openni::OpenNI::shutdown();
		return 2;
	} else {
		depthVideoMode = streamDepth.getVideoMode();
		m_width = depthVideoMode.getResolutionX();
		m_height = depthVideoMode.getResolutionY();
	}
	m_streams = new openni::VideoStream*[1];
	m_streams[0] = &streamDepth;

	return 0;
}
int m_device::close()
{
	int ret;

	g_RgbCap.release();

	streamDepth.stop();
	streamDepth.destroy();
	device.close();
	openni::OpenNI::shutdown();
	return 0;
}
int m_device::recvFrame(cv::Mat* mat_rgb, cv::Mat* mat_depth)
{

	cv::Mat frameRgb;
	g_RgbCap.read(frameRgb); // Capture a frame
	if (frameRgb.empty()) {
		std::cout << "Failed to read frame!" << std::endl;
		return -1;
	}
	*mat_rgb = frameRgb.clone();



	openni::VideoFrameRef frameDepth;
	int changedIndex;
	openni::Status rc = openni::OpenNI::waitForAnyStream(m_streams, 1, &changedIndex, SAMPLE_READ_WAIT_TIMEOUT);
	if (rc != openni::STATUS_OK) {
		printf("Wait failed\r\n");
		return -1;
	}
	rc = streamDepth.readFrame( &frameDepth );
	if (rc != openni::STATUS_OK) {
		printf("Read failed!\r\n");
		return -1;
	}

	int iMaxDepth = streamDepth.getMaxPixelValue();
	const cv::Mat mImageDepth( frameDepth.getHeight(), frameDepth.getWidth(), CV_16UC1, (void*)frameDepth.getData());
	cv::Mat mScaledDepth;
	mImageDepth.convertTo( mScaledDepth, CV_8U, 255.0 / iMaxDepth );

	cv::Mat flipped_horizontal;
	cv::flip(mScaledDepth, flipped_horizontal, 1);

	*mat_depth = flipped_horizontal.clone();


	return 0;
}

4.2 m_device.h

把m_device.h放到 ~/workspace/astrapro_out_of_box/include/

复制代码 
#ifndef __M_DEVICE_H
#define __M_DEVICE_H

#define device_inst            m_device::getInstance()


#include "stdint.h"                 // uint32_t

#include <opencv2/opencv.hpp>

class m_device {

public:
    m_device();
    virtual ~m_device();
    static m_device& getInstance();

    int init(int handle_index);
	int open();
    int close();
	int recvFrame(cv::Mat* mat_rgb, cv::Mat* mat_depth);

protected:

private:
};


#endif //__M_DEVICE_H

4.3 demo_device.cpp

把demo_device.cpp放到 ~/workspace/astrapro_out_of_box/demo

复制代码 
#include <stdio.h>
#include <unistd.h>         // usleep

#include "m_device.h"

int main(void)
{
	printf("[main]device_inst.init...\r\n");
	device_inst.init(0);
	printf("[main]device_inst.open...\r\n");
	device_inst.open();

	printf("[main]device_inst.recvFrame...\r\n");
	cv::Mat mat_rgb, mat_depth;
	int ret;
	while(1) {
		ret = device_inst.recvFrame(&mat_rgb, &mat_depth);
		if (ret>=0) {
			if (mat_rgb.empty()) {
				printf("Failed to read frame! \r\n");
				continue;
			}
			if (mat_depth.empty()) {
				printf("Failed to read frame! \r\n");
				continue;
			}
			imshow("mat_rgb", mat_rgb); 
			imshow("mat_depth", mat_depth); 
			if (cv::waitKey(1) == 'q') break; // Exit on 'q'
		}

	}
	printf("[main]device_inst.close...\r\n");
	device_inst.close();

//	cv::imwrite("mat_rgb.png", mat_rgb);
//	cv::imwrite("mat_depth.png", mat_depth);

//	cv::imshow("mat_rgb", mat_rgb);
//	cv::imshow("mat_depth", mat_depth);
//	cv::waitKey(0);
	cv::destroyAllWindows();

	return 0;
}

4.4 编译

4.4.1 目录结构

4.4.2 环境变量

复制代码 
# 环境变量
cd ~/workspace/astrapro_out_of_box
export WORK_DIR=$PWD
export ARCH="x64"

export C_COMPILER="gcc"
export CXX_COMPILER="g++"
export AR="ar"

4.4.3 用户源码编译出库

复制代码 
# 用户源码编译出库
$CXX_COMPILER -c src/*.cpp -Iinclude -Iext/install/$ARCH/include -Lext/install/$ARCH/lib
$AR -r libastra.a *.o
rm *.o

4.4.4 编译可执行程序

复制代码 
# 编译可执行程序
$CXX_COMPILER -o demo_device demo/demo_device.cpp -Iext/install/$ARCH/include -Iinclude -L. -Lext/install/$ARCH/lib  -lastra -lopencv_core -lopencv_highgui -lopencv_imgproc -lopencv_imgcodecs -lopencv_photo -lopencv_videoio -lopencv_video  -lOpenNI2

4.4.5 运行

复制代码 
# 运行
export WORK_DIR=$PWD
export ARCH="x64"
export LD_LIBRARY_PATH=$WORK_DIR/ext/install/$ARCH/lib:$LD_LIBRARY_PATH
./demo_device
相关推荐
电鱼智能的电小鱼6 小时前
基于电鱼 ARM 工控机的井下AI故障诊断方案——让煤矿远程监控更智能、更精准
网络·arm开发·人工智能·算法·边缘计算
拉姆哥的小屋6 小时前
时间序列早期分类中的置信度累积问题：从ECE-C到时序依赖建模
大数据·人工智能
蚁巡信息巡查系统6 小时前
政府网站与政务新媒体监测服务主要是做什么的？
大数据·人工智能
林恒smileZAZ6 小时前
移动端h5适配方案
人工智能·python·tensorflow
伟贤AI之路6 小时前
开源！纯 HTML 实现支持 0.75～2× 变速、iOS 熄屏防中断的英语点读站
人工智能·ai编程
编码时空的诗意行者6 小时前
LM实现教程：基于 nanochat项目 从零开始理解大语言模型
人工智能·语言模型·自然语言处理
兔兔爱学习兔兔爱学习6 小时前
ASR+MT+LLM+TTS 一体化实时翻译字幕系统
人工智能·自然语言处理·机器翻译
二向箔reverse6 小时前
用langchain搭建简单agent
人工智能·python·langchain
热门推荐
01GitHub 镜像站点02BongoCat - 跨平台键盘猫动画工具03UV安装并设置国内源04Linux下V2Ray安装配置指南05jdk21下载、安装（Windows、Linux、macOS）06NVIDIA显卡驱动、CUDA、cuDNN 和 TensorRT 版本匹配指南07安娜的档案(Anna’s Archive) 镜像网站/国内最新可访问入口（持续更新）08GitLab 零基础入门指南：从安装到项目管理全流程09KGG转MP3工具|非KGM文件|解密音频10在VSCode配置Java开发环境的保姆级教程（适配各类AI编程IDE）