ProxMox + VirtIO (KVM)

[Fr3DBr]

Hi, folks.

I'm trying to use pkt-gen in a virtual machine in proxmox.

Basically, I've loaded netmap.ko and virtio_net.ko drivers and I can confirm both are installed via lsmod and also by checking the driver version in ethtool showing as "-netmap".

Jan 29 16:56:04 MGXDEV kernel: 164.764164 [1492] netmap_finalize_obj_allocator Pre-allocated 1000000 clusters (4/4000000KB) for 'netmap_buf'
Jan 29 16:56:04 MGXDEV kernel: 164.764175 [ 433] netmap_init_obj_allocator_bitmap netmap_if free 256
Jan 29 16:56:04 MGXDEV kernel: 164.764181 [ 433] netmap_init_obj_allocator_bitmap netmap_ring free 1024
Jan 29 16:56:04 MGXDEV kernel: 164.766624 [ 433] netmap_init_obj_allocator_bitmap netmap_buf free 1000000
Jan 29 16:56:04 MGXDEV kernel: 164.766628 [1658] netmap_mem_finalize_all   interfaces 512 KB, rings 70016 KB, buffers 3906 MB
Jan 29 16:56:04 MGXDEV kernel: 164.766630 [1661] netmap_mem_finalize_all   Free buffers: 999998
Jan 29 16:56:04 MGXDEV kernel: 164.766643 [1989] netmap_interp_ringid      eth0: tx [0,48) rx [0,48) id 0
Jan 29 16:56:04 MGXDEV kernel: 164.768714 [ 493] virtio_net_netmap_detach_unused 1022 sgs detached on RX-0 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.769484 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-1 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.770200 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-2 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.770935 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-3 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.771672 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-4 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.772439 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-5 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.773162 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-6 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.773910 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-7 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.774638 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-8 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.775333 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-9 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.776065 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-10 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.776847 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-11 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.777977 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-12 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.778731 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-13 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.779436 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-14 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.780147 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-15 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.781126 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-16 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.781878 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-17 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.782588 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-18 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.783279 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-19 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.784017 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-20 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.785093 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-21 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.785918 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-22 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.786970 [ 493] virtio_net_netmap_detach_unused 1024 sgs detached on RX-23 (onoff=1)
Jan 29 16:56:04 MGXDEV kernel: 164.787165 [ 687] virtio_net_netmap_init_buffers eth0-rx-0: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787304 [ 687] virtio_net_netmap_init_buffers eth0-rx-1: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787462 [ 687] virtio_net_netmap_init_buffers eth0-rx-2: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787612 [ 687] virtio_net_netmap_init_buffers eth0-rx-3: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787752 [ 687] virtio_net_netmap_init_buffers eth0-rx-4: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787878 [ 687] virtio_net_netmap_init_buffers eth0-rx-5: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787942 [ 687] virtio_net_netmap_init_buffers eth0-rx-6: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.787998 [ 687] virtio_net_netmap_init_buffers eth0-rx-7: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788051 [ 687] virtio_net_netmap_init_buffers eth0-rx-8: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788107 [ 687] virtio_net_netmap_init_buffers eth0-rx-9: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788160 [ 687] virtio_net_netmap_init_buffers eth0-rx-10: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788214 [ 687] virtio_net_netmap_init_buffers eth0-rx-11: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788270 [ 687] virtio_net_netmap_init_buffers eth0-rx-12: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788325 [ 687] virtio_net_netmap_init_buffers eth0-rx-13: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788394 [ 687] virtio_net_netmap_init_buffers eth0-rx-14: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788450 [ 687] virtio_net_netmap_init_buffers eth0-rx-15: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788505 [ 687] virtio_net_netmap_init_buffers eth0-rx-16: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788558 [ 687] virtio_net_netmap_init_buffers eth0-rx-17: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788632 [ 687] virtio_net_netmap_init_buffers eth0-rx-18: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788689 [ 687] virtio_net_netmap_init_buffers eth0-rx-19: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788746 [ 687] virtio_net_netmap_init_buffers eth0-rx-20: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788799 [ 687] virtio_net_netmap_init_buffers eth0-rx-21: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788851 [ 687] virtio_net_netmap_init_buffers eth0-rx-22: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788904 [ 687] virtio_net_netmap_init_buffers eth0-rx-23: 1024 netmap buffers published
Jan 29 16:56:04 MGXDEV kernel: 164.788907 [ 624] netmap_set_all_rings      eth0: enable all rings
Jan 29 16:56:04 MGXDEV kernel: IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready

ethtool -i eth0
driver: virtio_net
version: 1.0.0-netmap
firmware-version: 
expansion-rom-version: 
bus-info: 0000:00:13.0
supports-statistics: no
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no

The thing is, when I'm using the netmap virtio driver, the packets are not sent to the network. While if I use the default virtio driver it works, but with a very poor performance (2 Mpps TX Max)

I've run a tcpdump in vmbr0 and I don't see any packets at all, so something else might be missing ?

Is there anything special that must be done, in order to use netmap+virtio in a regular proxmox (kvm) virtual machine ?

Thanks !

[Fr3DBr]

Hi @giuseppelettieri / @vmaffione do you have any hints here ? Thanks. :)