Non-nvidia limitations

[MauriceZ]

Hey just found this repo. Amazing work.

I'm working off of a crappy windows laptop that's dual-booting ubuntu. It does not have an nvidia GPU sadly.

What would I miss out on if i were to fork this and drop all features that require an nvidia GPU?

[JacopoPan]

Thanks @MauriceZ

there's nothing in the SITL simulation that CANNOT run purely on CPU (in fact, the main GitHub Action aas-amd64-build-and-test.yml runs headless on CPU-only runners) but Gazebo will likely struggle to keep even a 100% real-time factor if you want to use more than 1 drone and/or a complex scene.

I'd like to see such a fork and I can point you to where NVIDIA compute is explicitly used

1. You'll want to remove --device /dev/dri --gpus all and NVIDIA-related environmental variables

aerial-autonomy-stack/scripts/sim_run.sh

Lines 116 to 120 in d48cd97

	# Launch the simulation container
	DOCKER_CMD="docker run -it --rm \
	--volume /tmp/.X11-unix:/tmp/.X11-unix:rw --device /dev/dri --gpus all \
	--env DISPLAY=$DISPLAY --env QT_X11_NO_MITSHM=1 --env NVIDIA_DRIVER_CAPABILITIES=all --env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR --env GST_DEBUG=3 \
	--env __NV_PRIME_RENDER_OFFLOAD=1 --env __GLX_VENDOR_LIBRARY_NAME=nvidia \

aerial-autonomy-stack/scripts/sim_run.sh

Lines 149 to 153 in d48cd97

	# Launch the ground container
	DOCKER_CMD="docker run -it --rm \
	--volume /tmp/.X11-unix:/tmp/.X11-unix:rw --device /dev/dri --gpus all \
	--env DISPLAY=$DISPLAY --env QT_X11_NO_MITSHM=1 --env NVIDIA_DRIVER_CAPABILITIES=all --env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR --env GST_DEBUG=3 \
	--env __NV_PRIME_RENDER_OFFLOAD=1 --env __GLX_VENDOR_LIBRARY_NAME=nvidia \

aerial-autonomy-stack/scripts/sim_run.sh

Lines 182 to 186 in d48cd97

	local NAME_AIRCRAFT_CNT="aircraft-container-inst${INSTANCE}_${DRONE_ID}"
	DOCKER_CMD="docker run -it --rm \
	--volume /tmp/.X11-unix:/tmp/.X11-unix:rw --device /dev/dri --gpus all \
	--env DISPLAY=$DISPLAY --env QT_X11_NO_MITSHM=1 --env NVIDIA_DRIVER_CAPABILITIES=all --env XDG_RUNTIME_DIR=$XDG_RUNTIME_DIR --env GST_DEBUG=3 \
	--env __NV_PRIME_RENDER_OFFLOAD=1 --env __GLX_VENDOR_LIBRARY_NAME=nvidia \

2. In the simulation container, the gz to gst bridge should automatically switch to CPU encoding

aerial-autonomy-stack/simulation/simulation_resources/comms/gz_gst_bridge/gz_gst_bridge.cpp

Lines 92 to 104 in d48cd97

	if (check_nvidia_encoder()) {
	std::cout << "Using NVIDIA GPU Encoder (nvh264enc)\n";
	pipeline_str = "appsrc name=gz_source ! queue max-size-buffers=1 leaky=downstream ! "
	"videoconvert ! "
	"nvh264enc preset=low-latency-hq zerolatency=true rc-mode=cbr bitrate=2048 qp-min=15 qp-max=35 gop-size=30 ! "
	"rtph264pay config-interval=1 mtu=1400 ! udpsink sync=false " + ip_port;
	} else {
	std::cout << "Using CPU Encoder (x264enc)\n";
	pipeline_str = "appsrc name=gz_source ! queue max-size-buffers=1 leaky=downstream ! "
	"videoconvert ! "
	"x264enc speed-preset=ultrafast tune=zerolatency bitrate=2048 key-int-max=30 ! "
	"rtph264pay config-interval=1 mtu=1400 ! udpsink sync=false " + ip_port;
	}

3. In the aircraft container, ONNX Runtime should fail to load the CUDA provider (but likely default to the CPU provider automatically)

aerial-autonomy-stack/aircraft/aircraft_ws/src/yolo_py/yolo_py/yolo_node.py

Lines 159 to 170 in d48cd97

	if self.architecture == 'x86_64':
	max_dim = max(stream_width, stream_height)
	if max_dim <= 320:
	print("Stream resolution is <=320. Selecting 320 model.")
	model_path = "/aas/yolo/yolo26n_320.onnx"
	self.input_size = 320 # YOLO input size
	else:
	print("Stream resolution is >320. Selecting 640 model.")
	model_path = "/aas/yolo/yolo26n_640.onnx"
	self.input_size = 640 # YOLO input size
	print("Loading CUDAExecutionProvider on AMD64 (x86) architecture.")
	self.session = ort.InferenceSession(model_path, providers=["CUDAExecutionProvider"]) # For simulation