LION运行笔记

LION（Linear Group RNN for 3D Object Detection in Point Clouds）是华中科技大学白翔团队联合香港大学、百度提出的新一代 3D 物体检测框架，其核心创新在于将线性循环神经网络（Linear RNN）引入点云处理，显著提升了长距离依赖建模效率，同时降低了计算复杂度。

1. 环境安装

• 创建环境

conda create -n lion python=3.8

• 激活环境

conda activate lion

安装torch, torchvision, torchaudio

pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118

• 安装其它依赖项

把requirements.txt改成如下内容

dacite==1.8.1 
dask[dataframe]==2023.3.1 
einsum==0.3.0 
immutabledict==2.2.0 
jax==0.4.13 
jaxlib==0.4.13 
matplotlib==3.6.1 
pandas==1.5.3 
plotly==5.13.1 
pyarrow==16.0.0 
scikit-learn==1.2.2 
tensorflow_probability==0.21.0 
visu3d==1.5.1 
absl-py==1.4.0
setuptools==67.6.0
astunparse==1.6.3
attrs==23.2.0
Automat==22.10.0
buildtools==1.0.6
cachetools==5.2.0
causal-conv1d==1.2.0.post2
ccimport
charset-normalizer==3.3.2
constantly==23.10.4
cumm-cu118
docopt==0.6.2
dpcpp-cpp-rt==2024.0.3
easydict==1.13
einops==0.7.0
filelock==3.13.1
fire==0.6.0
flatbuffers>=2.0
fsspec==2024.3.1
furl==2.1.3
fvcore==0.1.5.post20221221
gast==0.4.0
google-auth==2.16.2
google-auth-oauthlib==0.4.6
google-pasta==0.2.0
greenlet==3.0.3
grpcio==1.48.2
h5py
huggingface-hub==0.21.4
hyperlink==21.0.0
idna==3.6
imageio==2.33.1
importlib_metadata==7.0.2
incremental==22.10.0
intel-cmplr-lib-rt==2024.0.3
intel-cmplr-lic-rt==2024.0.3
intel-opencl-rt==2024.0.3
intel-openmp==2024.0.3
iopath==0.1.10
Jinja2==3.1.3
keras-nightly==2.5.0.dev2021032900
Keras-Preprocessing
lark==1.1.9
lazy_loader==0.3
llvmlite==0.39.1
Markdown==3.4.1
MarkupSafe==2.1.1
mkl==2024.0.0
mkl-fft>=1.3.0
mkl-service==2.4.0
mpmath==1.3.0
networkx==3.1
ninja==1.11.1.1
numba==0.56.4
numpy==1.23.5
oauthlib==3.2.0
opt-einsum==3.3.0
orderedmultidict==1.0.1
packaging==24.0
pccm
pillow==9.2.0
portalocker==2.8.2
protobuf==3.20.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
pybind11==2.11.1
python-dateutil==2.9.0.post0
PyWavelets==1.4.1
PyYAML==6.0.1
redo==2.0.4
regex==2023.12.25
requests==2.31.0
requests-oauthlib==2.0.0
rsa==4.9
safetensors==0.4.2
scikit-image==0.20.0
scipy==1.9.1
SharedArray==3.0.0
simplejson==3.19.2
six==1.15.0
spconv-cu118==2.3.6
SQLAlchemy
sympy==1.12
tabulate==0.9.0
tbb==2021.11.0
tensorboard==2.12.0
tensorboard-data-server==0.7.0
tensorboard-plugin-wit==1.8.1
tensorboardX==2.6.2.2
tensorflow==2.12.0
tensorflow-estimator==2.12.0
termcolor==1.1.0
tifffile==2023.7.10
timm==0.9.16
tokenizers==0.15.2
torch-scatter==2.1.2
tqdm==4.66.2
transformers==4.38.2
triton==2.1.0
Twisted
typing_extensions==4.5.0
urllib3==2.2.1
Werkzeug==2.2.2
wrapt==1.12.1
yacs==0.1.8
zipp==3.18.1
zope.interface==6.2
kornia==0.5.8

pip install -r requirements.txt

• 安装LION

python setup.py develop

2. 数据准备

2.1. NuScenes Dataset

Please download the official NuScenes 3D object detection dataset and organize the downloaded files as follows:

OpenPCDet

├── data

│ ├── nuscenes

│ │ │── v1.0-trainval (or v1.0-mini if you use mini)

│ │ │ │── samples

│ │ │ │── sweeps

│ │ │ │── maps

│ │ │ │── v1.0-trainval

├── pcdet

├── tools

• Install the nuscenes-devkit with version 1.0.5 by running the following command:

pip install nuscenes-devkit==1.0.5

• Generate the data infos by running the following command (it may take several hours):

# for lidar-only setting
python -m pcdet.datasets.nuscenes.nuscenes_dataset --func create_nuscenes_infos \
    --cfg_file tools/cfgs/dataset_configs/nuscenes_dataset.yaml \
    --version v1.0-trainval

# for multi-modal setting
python -m pcdet.datasets.nuscenes.nuscenes_dataset --func create_nuscenes_infos \
    --cfg_file tools/cfgs/dataset_configs/nuscenes_dataset.yaml \
    --version v1.0-trainval \
    --with_cam

3. 训练

python tools/train.py --cfg_file tools/cfgs/lion_models/lion_mamba_nusc_8x_1f_1x_one_stride_128dim.yaml --extra_tag lion_mamba_nusc_8x_1f_1x_one_stride_128dim --batch_size 16 --epochs 36 --max_ckpt_save_num 4 --workers 4 --sync_bn

参考文献

https://github.com/open-mmlab/OpenPCDet?tab=readme-ov-file

https://github.com/happinesslz/LION/tree/main?tab=readme-ov-file