Discussion:
[pve-devel] pve-firewall : nftables ?
Alexandre DERUMIER
2018-11-26 08:00:47 UTC
Permalink
Hi,

I would like to known if somebody have already made some test with nftables recently ?

Mainly, is not possible to use physdev direction,

like:

-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT


I wonder if a simple vmap like this could work: ?

https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example


chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
Wolfgang Bumiller
2018-11-27 13:55:52 UTC
Permalink
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D

IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.

The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...
Josef Johansson
2018-11-27 13:58:31 UTC
Permalink
Post by Wolfgang Bumiller
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...
Wasn't nftables mostly iptables compatible?

Maybe it's a good thing to not freeze the current implementation.
Post by Wolfgang Bumiller
_______________________________________________
pve-devel mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--
Med vänliga hälsningar
Josef Johansson
Alexandre DERUMIER
2018-11-27 14:06:25 UTC
Permalink
Post by Wolfgang Bumiller
Post by Josef Johansson
Wasn't nftables mostly iptables compatible?
mostly :/


https://wiki.nftables.org/wiki-nftables/index.php/Nftables_families


Deprecated extensions
---------------------

physdev
br_netfilter aims to be deprecated by nftables.

quota
nfacct already provides quota support.

tos
deprecated by dscp




----- Mail original -----
De: "Josef Johansson" <***@oderland.se>
À: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 14:58:31
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Wolfgang Bumiller
Post by Josef Johansson
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...
Wasn't nftables mostly iptables compatible?

Maybe it's a good thing to not freeze the current implementation.
Post by Wolfgang Bumiller
_______________________________________________
pve-devel mailing list
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
--
Med vänliga hälsningar
Josef Johansson

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Alexandre DERUMIER
2018-11-27 14:19:41 UTC
Permalink
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
I have found an interesting paper from netdevconf 1.1 here:

https://www.netdevconf.org/1.1/proceedings/papers/Bridge-filter-with-nftables.pdf

I'll to do some tests in coming week.
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch...
yes, indeed. could be better to rewrite it from stratch. (and maybe better than iptables, with all the new features like vmap,...)
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The iptables part would be considered feature-frozen from that point on I'd say/hope/think...
Still missing some log features.

(Currently I can't have any log on an accept/reject rule)



----- Mail original -----
De: "Wolfgang Bumiller" <***@proxmox.com>
À: "Alexandre Derumier" <***@odiso.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 14:55:52
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D

IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.

The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...
Alexandre DERUMIER
2018-11-27 15:55:50 UTC
Permalink
Also,
it seem than conntrack is not yet implemented on bridge filtering :(

seem to be a blocking point for now


----- Mail original -----
De: "Alexandre Derumier" <***@odiso.com>
À: "Wolfgang Bumiller" <***@proxmox.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 15:19:41
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
I have found an interesting paper from netdevconf 1.1 here:

https://www.netdevconf.org/1.1/proceedings/papers/Bridge-filter-with-nftables.pdf

I'll to do some tests in coming week.
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch...
yes, indeed. could be better to rewrite it from stratch. (and maybe better than iptables, with all the new features like vmap,...)
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
The iptables part would be considered feature-frozen from that point on I'd say/hope/think...
Still missing some log features.

(Currently I can't have any log on an accept/reject rule)



----- Mail original -----
De: "Wolfgang Bumiller" <***@proxmox.com>
À: "Alexandre Derumier" <***@odiso.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 14:55:52
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D

IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.

The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Alexandre DERUMIER
2018-11-28 05:57:27 UTC
Permalink
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
Also,
it seem than conntrack is not yet implemented on bridge filtering :(
seem to be a blocking point for now
I have send a mail to the guy from the paper,

devs are currently working on bridge conntrack.

for now, it's possible to use a workaround, with

sysctl net.bridge.bridge-nf-call-iptables=1


then, something like this seem to work:

nft add table bridge filter
nft add chain bridge filter forward { type filter hook forward priority 0 \; }
nft add rule bridge filter forward log


nft add table filter
nft add chain filter forward { type filter hook forward priority 0 \; }
nft add rule filter forward ct state established,related counter accept


(don't have tested more than this, but i'm seeing vm connections in conntrack)



----- Mail original -----
De: "aderumier" <***@odiso.com>
À: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 16:55:50
Objet: Re: [pve-devel] pve-firewall : nftables ?

Also,
it seem than conntrack is not yet implemented on bridge filtering :(

seem to be a blocking point for now


----- Mail original -----
De: "Alexandre Derumier" <***@odiso.com>
À: "Wolfgang Bumiller" <***@proxmox.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 15:19:41
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
I have found an interesting paper from netdevconf 1.1 here:

https://www.netdevconf.org/1.1/proceedings/papers/Bridge-filter-with-nftables.pdf

I'll to do some tests in coming week.
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch...
yes, indeed. could be better to rewrite it from stratch. (and maybe better than iptables, with all the new features like vmap,...)
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The iptables part would be considered feature-frozen from that point on I'd say/hope/think...
Still missing some log features.

(Currently I can't have any log on an accept/reject rule)



----- Mail original -----
De: "Wolfgang Bumiller" <***@proxmox.com>
À: "Alexandre Derumier" <***@odiso.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 14:55:52
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D

IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.

The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Alexandre DERUMIER
2018-11-28 08:21:53 UTC
Permalink
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
sysctl net.bridge.bridge-nf-call-iptables=1
(don't have tested more than this, but i'm seeing vm connections in conntrack)
Damned, don't work because all is going to ip filter, and tap interface are physin/physout in this case
(and physdev is not supported by nftables)

:/

----- Mail original -----
De: "aderumier" <***@odiso.com>
À: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mercredi 28 Novembre 2018 06:57:27
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
Also,
it seem than conntrack is not yet implemented on bridge filtering :(
seem to be a blocking point for now
I have send a mail to the guy from the paper,

devs are currently working on bridge conntrack.

for now, it's possible to use a workaround, with

sysctl net.bridge.bridge-nf-call-iptables=1


then, something like this seem to work:

nft add table bridge filter
nft add chain bridge filter forward { type filter hook forward priority 0 \; }
nft add rule bridge filter forward log


nft add table filter
nft add chain filter forward { type filter hook forward priority 0 \; }
nft add rule filter forward ct state established,related counter accept


(don't have tested more than this, but i'm seeing vm connections in conntrack)



----- Mail original -----
De: "aderumier" <***@odiso.com>
À: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 16:55:50
Objet: Re: [pve-devel] pve-firewall : nftables ?

Also,
it seem than conntrack is not yet implemented on bridge filtering :(

seem to be a blocking point for now


----- Mail original -----
De: "Alexandre Derumier" <***@odiso.com>
À: "Wolfgang Bumiller" <***@proxmox.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 15:19:41
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D
I have found an interesting paper from netdevconf 1.1 here:

https://www.netdevconf.org/1.1/proceedings/papers/Bridge-filter-with-nftables.pdf

I'll to do some tests in coming week.
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch...
yes, indeed. could be better to rewrite it from stratch. (and maybe better than iptables, with all the new features like vmap,...)
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
The iptables part would be considered feature-frozen from that point on I'd say/hope/think...
Still missing some log features.

(Currently I can't have any log on an accept/reject rule)



----- Mail original -----
De: "Wolfgang Bumiller" <***@proxmox.com>
À: "Alexandre Derumier" <***@odiso.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mardi 27 Novembre 2018 14:55:52
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Hi,
I would like to known if somebody have already made some test with nftables recently ?
Mainly, is not possible to use physdev direction,
-A PVEFW-FWBR-OUT -m physdev --physdev-in tap160i1 --physdev-is-bridged -j tap160i1-OUT
I wonder if a simple vmap like this could work: ?
https://wiki.nftables.org/wiki-nftables/index.php/Classic_perimetral_firewall_example
chain forward {
type filter hook forward priority 0; policy drop;
jump global
oifname vmap { $nic_dmz : jump dmz_in , $nic_lan : jump lan_in }
oifname $nic_inet iifname vmap { $nic_dmz : jump dmz_out , $nic_lan : jump lan_out }
}
The issue was that the regular filter forward table isn't really used
for bridged traffic (IIRC?), while the bridge filter forward table
doesn't have access to conntrack. There may be other ways (at some
point I marked packets in the netdev tables), but I haven't checked in
a while.
At least I haven't produced any kernel crashes in a while ;-D

IIRC the issue with netdev tables on the other hand was that they'd have
to be created after a network device was created. Can't have them "wait
around" for the device. (Not a big deal, just needs a little more
callbacks in our interface creation code and lxc bridge hook.) I'd want
source mac & ip checks to be moved to those tables for outgoing
packets, they happen quite early in the stack.

The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel

_______________________________________________
pve-devel mailing list
pve-***@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Wolfgang Bumiller
2018-11-28 09:54:28 UTC
Permalink
Post by Alexandre DERUMIER
Post by Alexandre DERUMIER
sysctl net.bridge.bridge-nf-call-iptables=1
(don't have tested more than this, but i'm seeing vm connections in conntrack)
Damned, don't work because all is going to ip filter, and tap interface are physin/physout in this case
(and physdev is not supported by nftables)
I mean, it does "work™" if we keep the firewall bridges around, as we
can match on `fwbr404i0` etc...

But it would be nice if we could get rid of those...
I currently only see hacky workarounds for that, though... One way
would be to create indexed chains for each set of guest/interface pairs,
and use the netdev table's ingress hook to mark packets for them...
(And no, we can't simply mark the packets with the VMID itself because
that already eats up 30 of the 32 bits and there wouldn't be enough
space left for the interface number...)

We'd be using up more space in the 'mark' though (currently we only use
1 bit of it). But it would get rid of the extra firewall-bridges...

Like so:

table netdev PVE {
chain veth404i0 {
type filter hook ingress device veth404i0 priority 0; policy accept;
mark set 1
}
}

table inet filter pve {
chain Forward {
type filter hook forward priority 0;
ct state invalid drop
ct state {established, related} return
mark vmap {
1: jump Guest404i0,
2: jump Guest404i1,
...
}
}

chain Guest404i0 {
#put rules for vmid 404 interface 0 here...
}
chain Guest404i1 {
#put rules for vmid 404 interface 1 here...
}
}

I'd really like to just get the damn info... it's not like it's not
available for iptables already anyway -_- a `log` rule even prints all
of it ...
Alexandre DERUMIER
2018-11-28 11:03:23 UTC
Permalink
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
I mean, it does "work™" if we keep the firewall bridges around, as we
can match on `fwbr404i0` etc...
But it would be nice if we could get rid of those...
AFAIK, we also have added fwbr because we wanted the packet going twice in netfilter,
once for vm1 output
once for vm2 input

don't remember if it was for simplicity of rules matching,
like vm1 accept out tcp/80, vm2 reject in tcp/80, both vms on same bridge.

Need to look in my archives, but first patches was without fwbr.
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
But it would be nice if we could get rid of those...
I currently only see hacky workarounds for that, though... One way
would be to create indexed chains for each set of guest/interface pairs,
and use the netdev table's ingress hook to mark packets for them...
We'd be using up more space in the 'mark' though (currently we only use
1 bit of it). But it would get rid of the extra firewall-bridges...
oh, didn't known about netdev table
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
(And no, we can't simply mark the packets with the VMID itself because
that already eats up 30 of the 32 bits and there wouldn't be enough
space left for the interface number...)
:/
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
I'd really like to just get the damn info... it's not like it's not
available for iptables already anyway -_- a `log` rule even prints all
of it ...
which info ?


----- Mail original -----
De: "Wolfgang Bumiller" <***@proxmox.com>
À: "aderumier" <***@odiso.com>
Cc: "pve-devel" <pve-***@pve.proxmox.com>
Envoyé: Mercredi 28 Novembre 2018 10:54:28
Objet: Re: [pve-devel] pve-firewall : nftables ?
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
Post by Alexandre DERUMIER
sysctl net.bridge.bridge-nf-call-iptables=1
(don't have tested more than this, but i'm seeing vm connections in conntrack)
Damned, don't work because all is going to ip filter, and tap interface are physin/physout in this case
(and physdev is not supported by nftables)
I mean, it does "work™" if we keep the firewall bridges around, as we
can match on `fwbr404i0` etc...

But it would be nice if we could get rid of those...
I currently only see hacky workarounds for that, though... One way
would be to create indexed chains for each set of guest/interface pairs,
and use the netdev table's ingress hook to mark packets for them...
(And no, we can't simply mark the packets with the VMID itself because
that already eats up 30 of the 32 bits and there wouldn't be enough
space left for the interface number...)

We'd be using up more space in the 'mark' though (currently we only use
1 bit of it). But it would get rid of the extra firewall-bridges...

Like so:

table netdev PVE {
chain veth404i0 {
type filter hook ingress device veth404i0 priority 0; policy accept;
mark set 1
}
}

table inet filter pve {
chain Forward {
type filter hook forward priority 0;
ct state invalid drop
ct state {established, related} return
mark vmap {
1: jump Guest404i0,
2: jump Guest404i1,
...
}
}

chain Guest404i0 {
#put rules for vmid 404 interface 0 here...
}
chain Guest404i1 {
#put rules for vmid 404 interface 1 here...
}
}

I'd really like to just get the damn info... it's not like it's not
available for iptables already anyway -_- a `log` rule even prints all
of it ...
Wolfgang Bumiller
2018-11-28 11:44:06 UTC
Permalink
Post by Alexandre DERUMIER
Post by Wolfgang Bumiller
I mean, it does "work™" if we keep the firewall bridges around, as we
can match on `fwbr404i0` etc...
But it would be nice if we could get rid of those...
AFAIK, we also have added fwbr because we wanted the packet going twice in netfilter,
once for vm1 output
once for vm2 input
Right, we don't get the output vmid via the netdev table that way...
That's unfortunate... Maybe at postrouting... hmm. If only there were
'egress' hooks in the netdev chain as well.

Tom Weber
2018-11-28 10:06:23 UTC
Permalink
Post by Wolfgang Bumiller
The pve-firewall code is very iptables-oriented though, and I'm not sure
if maybe we're not better off splitting the rule-generating part out
and write the nftables variant from scratch... The iptables part would
be considered feature-frozen from that point on I'd say/hope/think...
Yes, I think in the long term rule generation really needs to be
separated completely from rule definition. Right now there's a lot of
implicit iptable rule generation inside pve-firewall, which makes it a
real pain.

Just to throw in another idea:
How about using something like shorewall (shorewall.net) to handle the
whole firewall generation code from a higher level. I'm using it for in
really complex setups for years and i am very happy with it. (I know
this won't solve the nftables problem right now).

  Tom
Dietmar Maurer
2018-11-28 10:56:37 UTC
Permalink
Post by Tom Weber
How about using something like shorewall (shorewall.net) to handle the
whole firewall generation code from a higher level. I'm using it for in
really complex setups for years and i am very happy with it. (I know
this won't solve the nftables problem right now).
No, I do not want to depend on such package.
Loading...