hi, after most parts works with my bridge setups works, I want to get vnet for my jails working. In the morning I started a jail and got only the local interface back, but no epair0b. Now I did something so that I can see _all_ interfaces from outside (bridge0 / bge* / epair0* ... ) but without any IPs. However, I'm not able to give epair0b inside the jail an ip address. I get "permission denied". Also it looks a bit strange: ==============host# jexec 2 ifconfig bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet autoselect (1000baseT <full-duplex>) status: active bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet autoselect (none) status: no carrier bge2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet autoselect (none) status: no carrier bge3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet autoselect (1000baseT <full-duplex>) status: active pflog0: flags=0<> metric 0 mtu 33152 ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536 ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 options=3<RXCSUM,TXCSUM> ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 12 priority 128 path cost 2000 member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> ifmaxaddr 0 port 4 priority 128 path cost 55 epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 options=8<VLAN_MTU> ether CHANGED ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) status: active ====================================== # host: jexec 2 ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up ifconfig: up: permission denied # sysctl: security.jail.enforce_statfs: 2 security.jail.mount_allowed: 0 security.jail.chflags_allowed: 0 security.jail.allow_raw_sockets: 1 security.jail.sysvipc_allowed: 1 security.jail.socket_unixiproute_only: 1 security.jail.set_hostname_allowed: 1 security.jail.jail_max_af_ips: 255 security.jail.jailed: 0 /etc/rc.conf: ============================jail_enable="YES" jail_v2_enable="YES" jail_list="" jail_sysvipc_allow="YES" #JAIL template jail_list="$jail_list template" jail_template_name="template" jail_template_hostname="template.CHANGED" jail_template_devfs_enable="YES" jail_template_rootdir="/jails/template" jail_template_mount_enable="YES" jail_template_fstab="/etc/jails/fstabs/template" jail_template_vnet_enable="YES" jail_template_devfs_ruleset="devfsrules_jail" #network jail_template_exec_prestart0="ifconfig epair0 create" jail_template_exec_prestart1="ifconfig bridge0 addm epair0a" jail_template_exec_prestart2="ifconfig epair0a up" jail_template_exec_earlypoststart0="ifconfig epair0b vnet template" jail_template_exec_afterstart0="ifconfig lo0 127.0.0.1" jail_template_exec_afterstart1="ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up" jail_template_exec_afterstart2="route add default 130.83.160.62" jail_template_exec_afterstart3="/bin/sh /etc/rc" jail_template_exec_poststop0="ifconfig bridge0 deletem epair0a" jail_template_exec_poststop1="ifconfig epair0a destroy" ========================== Starting jail: #/etc/rc.d/jail onestart Configuring jails:. Starting jails:epair0a ifconfig: up: permission denied route: writing to routing socket: Operation not permitted Setting hostname: example.mydomain.com. uname -a: 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue Jan 17 09:05:42 CET 2012 Also, some people say, I have to patch /etc/rc.d/jail (freeBSD 9-rc2) to get know the new "vnet2", other say, I don't need ... so .... Can anybody bring some light into the darkness of jails and vnet + rc? cu denny
I've done a bit of research about vnet jails: http://archive.0xfeedface.org/blog/2011-11-21/lattera/freebsd-vnet-jail-admin-project On Wed, Jan 18, 2012 at 6:59 AM, Denny Schierz <linuxmail@4lin.net> wrote:> hi, > > after most parts works with my bridge setups works, I want to get vnet for my jails working. In the morning I started a jail and got only the local interface back, but no epair0b. Now I did something so that I can see _all_ interfaces from outside (bridge0 / bge* / epair0* ... ) but without any IPs. > However, I'm not able to give epair0b inside the jail an ip address. I get "permission denied". > > Also ?it looks a bit strange: > > ==============> host# jexec 2 ifconfig > > bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=80099<RXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet autoselect (1000baseT <full-duplex>) > ? ? ? ?status: active > bge1: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet autoselect (none) > ? ? ? ?status: no carrier > bge2: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet autoselect (none) > ? ? ? ?status: no carrier > bge3: flags=8802<BROADCAST,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=8009b<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,VLAN_HWCSUM,LINKSTATE> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet autoselect (1000baseT <full-duplex>) > ? ? ? ?status: active > pflog0: flags=0<> metric 0 mtu 33152 > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ipfw0: flags=8801<UP,SIMPLEX,MULTICAST> metric 0 mtu 65536 > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> metric 0 mtu 16384 > ? ? ? ?options=3<RXCSUM,TXCSUM> > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > bridge0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?id 00:00:00:00:00:00 priority 32768 hellotime 2 fwddelay 15 > ? ? ? ?maxage 20 holdcnt 6 proto rstp maxaddr 100 timeout 1200 > ? ? ? ?root id 00:00:00:00:00:00 priority 32768 ifcost 0 port 0 > ? ? ? ?member: epair0a flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> > ? ? ? ? ? ? ? ?ifmaxaddr 0 port 12 priority 128 path cost 2000 > ? ? ? ?member: bge0 flags=143<LEARNING,DISCOVER,AUTOEDGE,AUTOPTP> > ? ? ? ? ? ? ? ?ifmaxaddr 0 port 4 priority 128 path cost 55 > epair0a: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=8<VLAN_MTU> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) > ? ? ? ?status: active > epair0b: flags=8842<BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ? ? ? ?options=8<VLAN_MTU> > ? ? ? ?ether CHANGED > ifconfig: socket(AF_INET6, SOCK_DGRAM): Protocol not supported > ? ? ? ?media: Ethernet 10Gbase-T (10Gbase-T <full-duplex>) > ? ? ? ?status: active > ======================================> > # host: > jexec 2 ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up > ifconfig: up: permission denied > > > > # sysctl: > > security.jail.enforce_statfs: 2 > security.jail.mount_allowed: 0 > security.jail.chflags_allowed: 0 > security.jail.allow_raw_sockets: 1 > security.jail.sysvipc_allowed: 1 > security.jail.socket_unixiproute_only: 1 > security.jail.set_hostname_allowed: 1 > security.jail.jail_max_af_ips: 255 > security.jail.jailed: 0 > > /etc/rc.conf: > ============================> jail_enable="YES" > jail_v2_enable="YES" > jail_list="" > jail_sysvipc_allow="YES" > > > #JAIL template > jail_list="$jail_list template" > jail_template_name="template" > jail_template_hostname="template.CHANGED" > jail_template_devfs_enable="YES" > jail_template_rootdir="/jails/template" > jail_template_mount_enable="YES" > jail_template_fstab="/etc/jails/fstabs/template" > jail_template_vnet_enable="YES" > jail_template_devfs_ruleset="devfsrules_jail" > > #network > jail_template_exec_prestart0="ifconfig epair0 create" > jail_template_exec_prestart1="ifconfig bridge0 addm epair0a" > jail_template_exec_prestart2="ifconfig epair0a up" > jail_template_exec_earlypoststart0="ifconfig epair0b vnet template" > jail_template_exec_afterstart0="ifconfig lo0 127.0.0.1" > jail_template_exec_afterstart1="ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up" > jail_template_exec_afterstart2="route add default 130.83.160.62" > jail_template_exec_afterstart3="/bin/sh /etc/rc" > jail_template_exec_poststop0="ifconfig bridge0 deletem epair0a" > jail_template_exec_poststop1="ifconfig epair0a destroy" > > ==========================> > Starting jail: > > #/etc/rc.d/jail onestart > > Configuring jails:. > Starting jails:epair0a > ifconfig: up: permission denied > route: writing to routing socket: Operation not permitted > Setting hostname: example.mydomain.com. > > uname -a: > > 9.0-STABLE FreeBSD 9.0-STABLE #0: Tue Jan 17 09:05:42 CET 2012 > > Also, some people say, I have to patch /etc/rc.d/jail (freeBSD 9-rc2) to get know the new "vnet2", other say, I don't need ... so .... > > Can anybody bring some light into the darkness of jails and vnet + rc? > > cu denny > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Hi, On 19/01/12 00:59, Denny Schierz wrote:> ======================================> > # host: > jexec 2 ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up > ifconfig: up: permission deniedI use 9.0.0 release for host and jail and a generic kernel with OPTIONS VIMAGE being the only change/addition. No problem.> > > # sysctl: > > security.jail.enforce_statfs: 2 > security.jail.mount_allowed: 0 > security.jail.chflags_allowed: 0 > security.jail.allow_raw_sockets: 1 > security.jail.sysvipc_allowed: 1security.jail.allow_raw_sockets: 0 security.jail.sysvipc_allowed: 0> security.jail.socket_unixiproute_only: 1 > security.jail.set_hostname_allowed: 1 > security.jail.jail_max_af_ips: 255 > security.jail.jailed: 0 > > /etc/rc.conf: > ============================> jail_enable="YES" > jail_v2_enable="YES" > jail_list="" > jail_sysvipc_allow="YES"I don't have this line, not sure what it does either.> > #JAIL template > jail_list="$jail_list template" > jail_template_name="template" > jail_template_hostname="template.CHANGED" > jail_template_devfs_enable="YES" > jail_template_rootdir="/jails/template" > jail_template_mount_enable="YES" > jail_template_fstab="/etc/jails/fstabs/template" > jail_template_vnet_enable="YES" > jail_template_devfs_ruleset="devfsrules_jail" > > #network > jail_template_exec_prestart0="ifconfig epair0 create" > jail_template_exec_prestart1="ifconfig bridge0 addm epair0a" > jail_template_exec_prestart2="ifconfig epair0a up" > jail_template_exec_earlypoststart0="ifconfig epair0b vnet template" > jail_template_exec_afterstart0="ifconfig lo0 127.0.0.1" > jail_template_exec_afterstart1="ifconfig epair0b 192.168.1.2 netmask 255.255.255.0 up" > jail_template_exec_afterstart2="route add default 130.83.160.62" > jail_template_exec_afterstart3="/bin/sh /etc/rc"^^^^^^^^^^^^^^^ The initscript runs /etc/rc already when the jail is created, this makes it run a second time which messed things up for me. When the initscript runs /etc/rc, all the scripts with NOJAIL are skipped. For some reason, when running /etc/rc a second time through this configuration, some daemons like cron were started a second time. I now execute a custom /etc/rc.jail which runs a few init scripts manually to configure networking and start a few daemons that don't come up with the original run of /etc/rc.> jail_template_exec_poststop0="ifconfig bridge0 deletem epair0a" > jail_template_exec_poststop1="ifconfig epair0a destroy"I am not doing this, because shortly after stopping a jail this would give me a kernel panic. If you comment those lines, the devices will stay where they are and simply be re-used the next time you start the jail again. Works without a problem for me. Regards Philipp