3D目标检测(室内)-TR3D
0 环境安装
TR3D是基于MMdetection的环境，但是环境依旧与MMdetection3D存在一些差异，我们可以利用docker或者参考docker来配置环境。如下是docker文件：
https://github.com/SamsungLabs/tr3d/blob/main/docker/Dockerfile
1 3D Detection-模型推理
命令行输入
- 测试数据(.bin文件)：/mnt/data1/mmdetection3d/data/scannet/scene000_00.bin
- 配置文件：configs/tr3d/tr3d_scannet-3d-18class.py
- 预训练模型权重参数文件：checkpoints/tr3d_scannet.pth
- 输出路径：–out-dir demo/scene0000_00/
- 检测得分阈值：–score-thr 0.6
- 是否可视化：–show
其中，配置文件在项目文件中均已给出，预训练模型权重参数文件需要手动下载一下。
对scannet数据集进行推理的代码如下：
# Example on scannet sample using tr3d model:
python demo/pcd_demo.py /mnt/data1/mmdetection3d/data/scannet/scene000_00.bin configs/tr3d/tr3d_scannet-3d-18class.py checkpoints/tr3d_scannet.pth --out-dir demo/scene0000_00/ --show --score-thr 0.6
但是，我们想利用预训练模型测试自己的数据，如何将自己的点云数据转为bin文件呢？在下一章中会介绍点云数据转换。
对于自己数据的模型推理，代码也是相似的，只需把数据(.bin)文件改成我们处理好了的自己数据的bin文件。
# Example on our own data using tr3d model:
python demo/data/own_data_generation.py
python demo/pcd_demo.py /mnt/data1/mmdetection3d/data/scannet/cloud_2hu.bin configs/tr3d/tr3d_scannet-3d-18class.py ../tr3d/checkpoints/tr3d_scannet.pth --out-dir demo/scene0000_00/ --show --score-thr 0.6
对于其他模型，我们也用类似的代码可以推理：
# Example on our own data using VoteNet model:
python demo/data/own_data_generation.py
python demo/pcd_demo.py /mnt/data1/mmdetection3d/data/scannet/cloud_2hu.bin configs/votenet/votenet_8x8_scannet-3d-18class.py checkpoints/votenet_8x8_scannet-3d-18class_20210823_234503-cf8134fa.pth --out-dir demo/scene0000_00/ --show --score-thr 0.5
2 点云数据转换
如下是scannet数据由npy转bin、ply转bin格式的代码。
转换代码其实很简单，但切记注意编码方式，float32还是float64！！！因为编码方式的错误，导致我在scannet能跑通的情况下，一直没法对自己的数据推理出预测结果，需要均转化为模型要求的float32。
import codecs
import sys
import numpy as np
from plyfile import PlyData, PlyElement
from os import path as osp
import chardet
import open3d as o3d
import pandas as pd
# 上面的库可能有些没用到
# ------------------------------------------------------------------------
# scannet由npy转bin格式
pts_filename = osp.join('/mnt/data1/mmdetection3d/data/scannet/' 'scannet_instance_data',
'scene0000_00_vert.npy')
points = np.load(pts_filename)
print(points)
print(points.shape)
# points[:,3] = points[:,3] / 255.0
# points[:,4] = points[:,4] / 255.0
# points[:,5] = points[:,5] / 255.0
# points[:,0] = (points[:,0] - min(points[:,0])) / (max(points[:,0] - min(points[:,0]))) * 9
# points[:,1] = (points[:,1] - min(points[:,1])) / (max(points[:,1] - min(points[:,1]))) * 9
# points[:,2] = (points[:,2] - min(points[:,2])) / (max(points[:,2] - min(points[:,2]))) * 3
# tmp = points[:,0].copy()
# points[:,0] = points[:,1]
# points[:,1] = tmp
np.savetxt('./scene0000_00_vert.txt', points, fmt='%.3f %.3f %.3f %d %d %d')
points.tofile(osp.join('/mnt/data1/mmdetection3d/data/scannet/','scene0000_00.bin'))
# 推理命令(可行): python demo/pcd_demo.py /mnt/data1/mmdetection3d/data/scannet/scene0000_00.bin
# configs/votenet/votenet_8x8_scannet-3d-18class.py checkpoints/votenet_8x8_scannet-3d-18class_20210823_234503-cf8134fa.pth
# --out-dir demo/scene0000_00/ --show --score-thr 0.5
# ------------------------------------------------------------------------
# ------------------------------------------------------------------------
# # 读取bin
# with open(osp.join('/mnt/data1/mmdetection3d/data/scannet/','scene0000_00.bin'), 'rb') as f:
# # 读取文件内容
# binary_data = f.read()
# # 将二进制数据解析为NumPy数组
# bin_array = np.frombuffer(binary_data, dtype=np.float32)
# print(bin_array)
# print(bin_array.shape)
# ------------------------------------------------------------------------
# ------------------------------------------------------------------------
# scannet由ply转bin格式(cut)
# pcd = o3d.io.read_point_cloud('/mnt/data1/tr3d/scene0000_00_cut.ply')
# xyz_load = np.asarray(pcd.points)
# colors_load = np.asarray(pcd.colors)
# points = np.hstack((xyz_load, colors_load * 255))
# print(points.shape)
# print(pd.Series(points[:,4]).describe())
# o3d.visualization.draw_geometries([pcd])
# points.tofile(osp.join('/mnt/data1/mmdetection3d/data/scannet/','scene0000_00_cut.bin'))
# ------------------------------------------------------------------------
# ------------------------------------------------------------------------
# cloud.ply转npy，转bin格式
pcd = o3d.io.read_point_cloud('./cloud_2hu.ply')
xyz_load = np.asarray(pcd.points)
colors_load = np.asarray(pcd.colors)
points2 = np.hstack((xyz_load, colors_load * 255))
# points2 = np.hstack((xyz_load, colors_load))
points2[:,0] = (points2[:,0] - min(points2[:,0])) / (max(points2[:,0] - min(points2[:,0]))) * 9
points2[:,1] = (points2[:,1] - min(points2[:,1])) / (max(points2[:,1] - min(points2[:,1]))) * 9
points2[:,2] = (points2[:,2] - min(points2[:,2])) / (max(points2[:,2] - min(points2[:,2]))) * 3
# print(points2)
print(pd.Series(points2[:,3]).describe())
print(pd.Series(points[:,3]).describe())
print(pd.Series(points2[:,4]).describe())
print(pd.Series(points[:,4]).describe())
print(pd.Series(points2[:,5]).describe())
print(pd.Series(points[:,5]).describe())
print(pd.Series(points2[:,0]).describe())
print(pd.Series(points[:,0]).describe())
print(pd.Series(points2[:,1]).describe())
print(pd.Series(points[:,1]).describe())
print(pd.Series(points2[:,2]).describe())
print(pd.Series(points[:,2]).describe())
o3d.visualization.draw_geometries([pcd])
np.savetxt('./cloud4.txt', points2, fmt='%.3f %.3f %.3f %d %d %d')
points2 = points2.astype('float32') # 一定要转成float32，否则怎么跑都跑不出结果
points2.tofile(osp.join('/mnt/data1/mmdetection3d/data/scannet/','cloud_2hu.bin'))
# 推理命令(输出为空): python demo/pcd_demo.py /mnt/data1/mmdetection3d/data/scannet/cloud3.bin
# configs/votenet/votenet_8x8_scannet-3d-18class.py checkpoints/votenet_8x8_scannet-3d-18class_20210823_234503-cf8134fa.pth
# --out-dir demo/scene0000_00/ --show --score-thr 0
# ------------------------------------------------------------------------