netlink List implementation

[aojea]

Implement List() method using netlink directly to avoid the overhead of
calling the nft user space progam and parsing its output. Also, reducing
the exposure to bugs or problems caused by skew versions of the binary
when running inside containers.

[danwinship]

To get further improvements, we would need to reduce pointers, and
change knftables.Rule to use values instead of pointers for Handle, Index, etc.
This would save millions of allocations at scale but is a breaking change.

In general if you are listing large numbers of rules, you're probably doing something wrong anyway, in terms of good nftables architecture. (kube-proxy never calls ListRules. Calico does, and I haven't looked into exactly what they're doing, but I suspect it's mostly because they were trying to make the nftables backend work roughly the same way as the iptables backend.)

We could optimize the Handle allocation without breaking API by allocating a single []int along with the []*Rule and allocating all of the handles out of that. (Though you'd want to fix the code to pre-allocate those arrays to the right length first in that case to avoid blocking GC of every intermediate version of the handle array.)

This would be less memory-efficient than the current system in the case where someone lists a lot of rules and then keeps a handle to just one of them, but again, "probably doing something wrong anyway".

(Despite what the comment says, neither the old nor the new code sets Index from ListRules)

[aojea]

@danwinship what do we want to do with this? do we wait?

[danwinship]

I think we should backport the --terse fix to 1.35 and 1.34, and then work on netlink List() support (and maybe more) for 1.37.

[aojea]

I see you already tagged itv0.0.21 with the terse fix

[aojea]

@danwinship finally get to it, addressed your comments and bumped dependencies, there is some work needed to fix the jobs with the golang update

[danwinship]

should we be doing the .go-version thing like other repos?

[danwinship]

/lgtm
/approve

we can do .go-version later if we want

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: aojea, danwinship

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [danwinship]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment