thr3ads.net - freebsd stable - Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade [Apr 2018]

If this information is useful, please help other people find it:
Share via:

Marc Branchaud

2018-Apr-05 14:28 UTC

Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade

Hi all,

I just upgraded from 10.3 to 10.4, and "/etc/rc.d/jail start" is
having
problems starting all of my jails:

# /etc/rc.d/jail start
Starting jails:xipbuild_3_3: created
ifconfig:: bad value
jail: xipbuild_3_3_8: /sbin/ifconfig lo1 inet 10.1.1.38/32 alias: failed
xipbuild_3_4: created
ifconfig:: bad value
jail: xipbuild_4_0: /sbin/ifconfig lo1 inet 10.1.1.5/32 alias: failed
xipbuild: created
xipbuild_4_9: created
ifconfig:: bad value
jail: xipbuild9: /sbin/ifconfig lo1 inet 10.1.1.209/32 alias: failed
.

This worked fine in 10.3.  I can individually start each jail, e.g. 
"/etc/rc.d/jail start xipbuild9".

All the jails configure the same set of parameters.  Here's my jail.conf:

--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---
xipbuild_3_3 {
   path="/usr/build-jails/jails/3.3";
   host.hostname="xipbuild_3_3";
   ip4.addr="10.1.1.3/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/3.3/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild_3_3_8 {
   path="/usr/build-jails/jails/3.3.8";
   host.hostname="xipbuild_3_3_8";
   ip4.addr="10.1.1.38/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/3.3.8/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild_3_4 {
   path="/usr/build-jails/jails/3.4";
   host.hostname="xipbuild_3_4";
   ip4.addr="10.1.1.4/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/3.4/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild_4_0 {
   path="/usr/build-jails/jails/4.0";
   host.hostname="xipbuild_4_0";
   ip4.addr="10.1.1.5/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/4.0/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild {
   path="/usr/build-jails/jails/latest";
   host.hostname="xipbuild";
   ip4.addr="10.1.1.200/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/latest/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild_4_9 {
   path="/usr/build-jails/jails/4.9";
   host.hostname="xipbuild_4_9";
   ip4.addr="10.1.1.90/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/4.9/usr/home nullfs rw 0
0";
   interface="lo1";
}
xipbuild9 {
   path="/usr/build-jails/jails/latest9";
   host.hostname="xipbuild9";
   ip4.addr="10.1.1.209/32";

   allow.chflags;
   allow.mount;
   mount.devfs;

   persist;

   mount="/usr/home  /usr/build-jails/jails/latest9/usr/home nullfs rw 0
0";
   interface="lo1";
}
--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---

I use ipnat to give the jails network access.  Here's ipnat.rules:

--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---
map em0 10.1.1.0/24 -> 0/32 proxy port ftp ftp/tcp
map em0 10.1.1.0/24 -> 0/32
--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---

And here's my rc.conf:

--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---
# Generated by Ansible

# hostname must be FQDN
hostname="devastator.xiplink.com"

zfs_enable="False"

# FIXME: previously auto-created?
ifconfig_lo1="create"


ifconfig_em0="DHCP SYNCDHCP"

network_interfaces="em0"
gateway_enable="YES"

# Prevent rpc
rpcbind_enable="NO"

# Prevent sendmail to try to connect to localhost
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Bring up sshd, it takes some time and uses some entropy on first startup
sshd_enable="YES"

netwait_enable="YES"
netwait_ip="10.10.0.35"
netwait_if="em0"

jenkins_swarm_enable="YES"
jenkins_swarm_opts="-executors 8"

# --- Build jails ---
build_jails_enable="YES"
jail_enable="YES"

# Set rules in /etc/ipnat.rules
ipnat_enable="YES"

# Set interface name for ipnat
network_interfaces="${network_interfaces} lo1"

# Each jail needs to specify its IP address and mask bits in ipv4_addrs_lo1
ipv4_addrs_lo1="10.1.1.1/32"

jail_chflags_allow="yes"

varmfs="NO"
--- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8<
---

Any insight would be deeply appreciated!

		M.

Marc Branchaud

2018-Apr-06 18:26 UTC

head link

Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade

On 2018-04-05 10:28 AM, Marc Branchaud wrote:> Hi all,
> 
> I just upgraded from 10.3 to 10.4, and "/etc/rc.d/jail start" is
having
> problems starting all of my jails:
> 
> # /etc/rc.d/jail start
> Starting jails:xipbuild_3_3: created
> ifconfig:: bad value
> jail: xipbuild_3_3_8: /sbin/ifconfig lo1 inet 10.1.1.38/32 alias: failed
> xipbuild_3_4: created
> ifconfig:: bad value
> jail: xipbuild_4_0: /sbin/ifconfig lo1 inet 10.1.1.5/32 alias: failed
> xipbuild: created
> xipbuild_4_9: created
> ifconfig:: bad value
> jail: xipbuild9: /sbin/ifconfig lo1 inet 10.1.1.209/32 alias: failed
> .
More info: Things work fine with jail_parallel_start="YES".

In 10.4, /etc/rc.d/jail now adds "-p1" to the jail command's
arguments
when starting all jails with jail_parallel_start="NO".  It's
definitely
this parameter that's causing my problems -- changing /etc/rc.d/jail to 
not add the parameter fixes the problem.

		M.

> This worked fine in 10.3.? I can individually start each jail, e.g. 
> "/etc/rc.d/jail start xipbuild9".
> 
> All the jails configure the same set of parameters.? Here's my
jail.conf:
> 
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> xipbuild_3_3 {
>  ? path="/usr/build-jails/jails/3.3";
>  ? host.hostname="xipbuild_3_3";
>  ? ip4.addr="10.1.1.3/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/3.3/usr/home nullfs rw 0
0";
>  ? interface="lo1";
> }
> xipbuild_3_3_8 {
>  ? path="/usr/build-jails/jails/3.3.8";
>  ? host.hostname="xipbuild_3_3_8";
>  ? ip4.addr="10.1.1.38/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/3.3.8/usr/home nullfs rw 0
0";
>  ? interface="lo1";
> }
> xipbuild_3_4 {
>  ? path="/usr/build-jails/jails/3.4";
>  ? host.hostname="xipbuild_3_4";
>  ? ip4.addr="10.1.1.4/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/3.4/usr/home nullfs rw 0
0";
>  ? interface="lo1";
> }
> xipbuild_4_0 {
>  ? path="/usr/build-jails/jails/4.0";
>  ? host.hostname="xipbuild_4_0";
>  ? ip4.addr="10.1.1.5/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/4.0/usr/home nullfs rw 0
0";
>  ? interface="lo1";
> }
> xipbuild {
>  ? path="/usr/build-jails/jails/latest";
>  ? host.hostname="xipbuild";
>  ? ip4.addr="10.1.1.200/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/latest/usr/home nullfs rw
0 0";
>  ? interface="lo1";
> }
> xipbuild_4_9 {
>  ? path="/usr/build-jails/jails/4.9";
>  ? host.hostname="xipbuild_4_9";
>  ? ip4.addr="10.1.1.90/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/4.9/usr/home nullfs rw 0
0";
>  ? interface="lo1";
> }
> xipbuild9 {
>  ? path="/usr/build-jails/jails/latest9";
>  ? host.hostname="xipbuild9";
>  ? ip4.addr="10.1.1.209/32";
> 
>  ? allow.chflags;
>  ? allow.mount;
>  ? mount.devfs;
> 
>  ? persist;
> 
>  ? mount="/usr/home? /usr/build-jails/jails/latest9/usr/home nullfs rw
0
> 0";
>  ? interface="lo1";
> }
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> 
> I use ipnat to give the jails network access.? Here's ipnat.rules:
> 
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> map em0 10.1.1.0/24 -> 0/32 proxy port ftp ftp/tcp
> map em0 10.1.1.0/24 -> 0/32
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> 
> And here's my rc.conf:
> 
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> # Generated by Ansible
> 
> # hostname must be FQDN
> hostname="devastator.xiplink.com"
> 
> zfs_enable="False"
> 
> # FIXME: previously auto-created?
> ifconfig_lo1="create"
> 
> 
> ifconfig_em0="DHCP SYNCDHCP"
> 
> network_interfaces="em0"
> gateway_enable="YES"
> 
> # Prevent rpc
> rpcbind_enable="NO"
> 
> # Prevent sendmail to try to connect to localhost
> sendmail_enable="NO"
> sendmail_submit_enable="NO"
> sendmail_outbound_enable="NO"
> sendmail_msp_queue_enable="NO"
> 
> # Bring up sshd, it takes some time and uses some entropy on first startup
> sshd_enable="YES"
> 
> netwait_enable="YES"
> netwait_ip="10.10.0.35"
> netwait_if="em0"
> 
> jenkins_swarm_enable="YES"
> jenkins_swarm_opts="-executors 8"
> 
> # --- Build jails ---
> build_jails_enable="YES"
> jail_enable="YES"
> 
> # Set rules in /etc/ipnat.rules
> ipnat_enable="YES"
> 
> # Set interface name for ipnat
> network_interfaces="${network_interfaces} lo1"
> 
> # Each jail needs to specify its IP address and mask bits in ipv4_addrs_lo1
> ipv4_addrs_lo1="10.1.1.1/32"
> 
> jail_chflags_allow="yes"
> 
> varmfs="NO"
> --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< --- 8< ---
8< ---
> 
> Any insight would be deeply appreciated!
> 
>  ??????? M.

freebsd stable - Apr 2018 - Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade

Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade

Problems with ifconfig when starting all jails after 10.3 -> 10.4 upgrade