Use of netmaker for configuring fully meshed wireguard vpn

[dfry]

Hello, great project by the way.

I am interested in knowing why Kairos has chosen to implement a meshed vpn solution based on edgevpn as opposed to leveraging a wireguard-based solution such as netmaker: https://github.com/gravitl/netmaker

The fact that edgevpn seems to indicate that it is not suitable for production seems deserving of a mention when advertising "Secure peer to peer mesh with VPN" as a feature on kairos.io's main page.

From edgevpn's github page:
⚠️ Warning!
I'm not a security expert, and this software didn't went through a full security audit, so don't use and rely on it for sensible traffic and not even for production environment! I did this mostly for fun while I was experimenting with libp2p.

[mudler]

Hi @dfry 👋

Netmaker is a relatively young project (https://github.com/gravitl/netmaker/releases/tag/v0.1). EdgeVPN first takes goes back to 2020, where NetMaker wasn't a choice (yet) and Wireguard was not available as a module in the Linux Kernel. Note that our documentation refer to this implementation as "Experimental". It is secure, as it connections are still e2e encrypted between nodes, and, of course, it's an additional layer on top of plain traffic.

However, while NetMaker seems great for VPNs connections, EdgeVPN does more than that - it actually spans as co-ordination layer with a shared ledger - as such is used by Kairos to organize nodes, but - if you don't need a VPN - it uses it only as a layer for consesus and set-up nodes autonomously. Most importantly, doesn't require a central server - at all, and it is by design.

I'd suggest to have a look at https://kairos.io/docs/architecture/network/ as describes the rationale, and how it is used. It also gives some insights in how does it work.

As you noted, I'm not a network security expert and the software didn't went through a full security audit so I'm being transparent there, however between peers review the architecture seemed very solid - did netmaker went to a full security audit that is publicly accessible, so I can compare?

[dfry]

Hi @mudler , thanks for the quick reply. I am not a security expert either, so I was just asking if thought had been put in to alternatives, which you just answered. By the way, I just pasted the warning message you have on the edgevpn github page verbatim, I have no connection to the netmaker project and do not know if it is more secure or had a full security audit performed. I have been using wireguard for a few years on my projects and have been trying to integrate netmaker into my infrastructure over the last couple of months. Now that I am looking at leveraging kairos, would love to use netmaker as a integrated feature instead of reinventing the wheel.

In any case, It is my 2nd day reading up on kairos, I will take your suggestion and take a deep dive into the architecture docs.

Best regards

[mudler]

makes sense! I'm just asking because it is a topic that came several times, so I'm happy to consider alternatives that can cover the same features that EdgeVPN offers, plus gives more confidence on the security aspect (if it went through an audit, I'm even more than happy to include it).

I think there is nothing stopping using Netmaker in place of EdgeVPN. EdgeVPN is opt-in, and if you don't need any of the automation from it, you can just ignore it!

I've seen other people in the community using Tailscale with Kairos just fine. One way to integrate external services is via Kairos bundle mechanism: https://kairos.io/docs/advanced/bundles/

Hope it helps!

[dfry]

thanks for the tips @mudler , I am looking forward to using kairos.

[infogulch]

It sounds like the critical feature EdgeVPN provides is automatic node setup and overlay network creation without a central server. This is important because kairos can't assume access to a central server when the nodes are coming up (e.g. an edge with no general internet connection, or a homelab where these are the only servers, etc), but they still need to coordinate to form a mesh.

There's nothing about this feature that excludes using wireguard as the point to point transport technology. E.g. this is what wesher does (though I'm not necessarily recommending this because it appears to be even less mature than netmaker was).

The latency and efficiency benefits that kernel wireguard offers over userspace would be a very nice bonus for low powered edge devices and high throughput nodes alike. Have you considered integrating wireguard with EdgeVPN?