My first attempt seems to be empty when its arrived at the maillinglist, so i
try it again in pure text format (i hope). Sorry, if there are unwanted
duplicates.
Since a few days i was experimenting with jails build from pkgbase (i got the
idea from Michael W. Lucas Freebsd Jails book). Now there is some discussion
about pkgbase again :)
So i want to share my experience, i hope this is not offtopic to this
conversation and my writing style is not too offputting...
?
This experiments were made on an resently updated 12-stable system. Sorry, i
dont have a system with head?available right now.
?
Anyway, this is what i did:
- I first set the variable "REPODIR=/usr/repo" in /etc/make.conf, to
have a nonvolatile repository (is this the right place/variable to do this?).
- then after the standard "make
buildworld|installworld|buildkernel|installkernel" in /usr/src, i created
the pkgbase packages with the command "make packages".
? the files are placed in "/usr/repo/FreeBSD:12:amd64/latest" where
latest is a symlink to the, well latest build.
? drwxr-xr-x ?2 root ?wheel ?805 14 Apr. 14:59 12.0.s20190414123806/
? drwxr-xr-x ?2 root ?wheel ?805 21 Apr. 09:27 12.0.s20190419155807/
? drwxr-xr-x ?2 root ?wheel ?805 22 Apr. 13:53 12.0.s20190422094219/
? drwxr-xr-x ?2 root ?wheel ?805 27 Apr. 14:12 12.0.s20190427084356/
? lrwxr-xr-x ?1 root ?wheel ? 20 27 Apr. 11:04 latest@ ->
12.0.s20190427084356
??
- this repo is published via the file
"/usr/local/etc/pkg/repos/FreeBSD-base.conf"which has this content:
root at betablock:~ # cat "/usr/local/etc/pkg/repos/FreeBSD-base.conf"
# FreeBSD base system repository
FreeBSD-base: {
? url: "file:///usr/repo/${ABI}/latest",
? mirror_type: "none",
? enabled: yes
}
?
now to the jail stuff:
- this command installs the FreeBSD-runtime and the 72 neccesary dependencies in
the jails root directory /jails/test03:
root at betablock:~ # pkg --rootdir /jails/test03 -o
'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-runtime
?
- the newly created jail "test03" (an entry in /etc/jail.conf already
exists) is started with the command:
root at betablock:~ # jail -vc test03
?
sadly the jail wont start, it exits with this message:
...
test03: created
test03: run command in jail: /bin/sh /etc/rc
jail: test03: getpwnam: No such file or directory
jail: test03: /bin/sh /etc/rc: failed
test03: removed
...
?
This happens, because there are no user-database files (like master.passwd,
pwd.db, spwd.db) and no group files in the jails /etc/ directory. hmmm...
I saw in the pkg install output of the FreeBSD-runtime ?pkg this message:
...
[2/72] Extracting FreeBSD-runtime-12.0.s20190414123806: 100%
pwd_mkdb: /jails/test03/etc/master.passwd: No such file or directory
...
so pkg tries to run the pwd_mkdb as defined in the runtime.ucl file, or embedded
in the pkg file, it is viewable with this command:
root at betablock:~ # pkg info --raw --file
/usr/repo/FreeBSD:12:amd64/latest/FreeBSD-runtime-12.0.s20190414123806.txz
...
scripts: {
? ? post-install: "cap_mkdb -l ${PKG_ROOTDIR}/etc/login.conf\n\tpwd_mkdb -i
-p -d ?${PKG_ROOTDIR}/etc ${PKG_ROOTDIR}/etc/master.passwd\n\tservices_mkdb -l
-q -o ${PKG_ROOTDIR}/var/db/services.db ${PKG_ROOTDIR}/etc/services\n\tchmod
1777 ${PKG_ROOTDIR}/tmp"
}
...
- so for a quick fix, i copied the "missing" files (master.passwd,
pwd.db, spwd.db, group) from a FreeBSD base.txz package to my jails etc
directory and tried to start the jail again. Read below for a "better"
solution.
Now the jail starts:
root at betablock:~ # jls
? ? ?JID ?IP Address ? ? ?Hostname ? ? ? ? ? ? ? ? ? ? ?Path
? ? ?...
? ? ?5 ?192.168.0.113 ? test03.local ? ? ? ? ? ? ? ? ?/jails/test03
?
i can "enter" it with this command:
root at betablock:~ # jexec -l test03
??
now to some more experiments with the jail which results in some questions
(also, see below)
?
- trying to ping something:
root at test03:~ # ping google.com
ld-elf.so.1: Shared object "libcap_dns.so.1" not found, required by
"ping"
?
the required library libcap_dns.so.1 is in the file
FreeBSD-libcasper-12.0.s20190414123806.txz, so lets install it:
Outside of the jail:
root at betablock:~ # pkg --rootdir /jails/test03 -o
'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-libcasper
?
and inside the jail:
root at test03:~ # ping google.com
PING google.com (216.58.213.206): 56 data bytes
64 bytes from 216.58.213.206: icmp_seq=0 ttl=52 time=23.398 ms
64 bytes from 216.58.213.206: icmp_seq=1 ttl=52 time=23.975 ms
64 bytes from 216.58.213.206: icmp_seq=2 ttl=52 time=23.252 ms
...
success!
?
The size of this freshly installed jail:
du -h -d 1 /jails/
...
90M ? ?/jails/test03
?
- now lets install some packages from outside of the jail:
root at betablock:~ # pkg --rootdir /jails/test03 install nginx-full
Updating FreeBSD repository catalogue...
pkg: Repository FreeBSD load error: access repo
file(/jails/test03/var/db/pkg/repo-FreeBSD.sqlite) failed: No such file or
directory
Fetching meta.txz: 100% ? ?944 B ? 0.9kB/s ? ?00:01
Fetching packagesite.txz: 100% ? ?6 MiB ? 2.2MB/s ? ?00:03
Processing entries: 100%
FreeBSD repository update completed. 31883 packages processed.
Updating Synth repository catalogue...
Synth repository is up to date.
Updating FreeBSD-base repository catalogue...
FreeBSD-base repository is up to date.
All repositories are up to date.
The following 146 package(s) will be affected (of 0 checked):
New packages to be INSTALLED:
? ? ? ? nginx-full: 1.16.0_2,2 [FreeBSD]
? ? ? ? apache24: 2.4.39 [FreeBSD]
? ? ? ? libnghttp2: 1.38.0 [FreeBSD]
...
? ? ? ? libdrizzle: 0.8_6 [FreeBSD]
? ? ? ? brotli: 1.0.7_1,1 [FreeBSD]
Number of packages to be installed: 146
The process will require 953 MiB more space.
232 MiB to be downloaded.
...
?
Funfact -?the jailsize after this installation:
1,0G ? ?/jails/test03
?
- lets enable the nginx service
root at betablock:~ # sysrc -R /jails/test03 nginx_enable=YES
nginx_enable: ?-> YES
?
- and start it (after a jail restart, without it nginx does not find its
libraries, installed above)
root at test03:~ # service nginx start
Performing sanity check on nginx configuration:
ld-elf.so.1: Shared object "libexecinfo.so.1" not found, required by
"libprofiler.so.0"
?
- lets install this missing library with the command?
root at betablock:~ # pkg --rootdir /jails/test03 -o
'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-libexecinfo
?
- and lets also install vi:
root at betablock:~ # pkg --rootdir /jails/test03 -o
'ASSUME_ALWAYS_YES=true' install -r FreeBSD-base FreeBSD-vi
?
- so lets try again (after creating the missing config files nginx.conf and
mime.types):
root at betablock:~ # jexec -l test03 service nginx start
Starting nginx.
nginx: [emerg] gethostbyname() failed in /usr/local/etc/nginx/nginx.conf:122
?
- and the check:
root at betablock:~ # jexec -l test03 service nginx status
nginx is running as pid 41359.
?
(Browser access works too...)
?
- Now Lets work with packages within the jail:
root at test03:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest,
please wait...
pkg: Error loading revoked certificates
?
- Hm, this is because the directory "/usr/share/keys/pkg/revoked/" is
missing, lets create it:
root at test03:~ # mkdir /usr/share/keys/pkg/revoked/
?
- and now:
root at test03:~ # pkg update
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from
pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest[http://pkg.FreeBSD.org/FreeBSD:12:amd64/latest],
please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[test03.local] Installing pkg-1.10.5_5...
[test03.local] Extracting pkg-1.10.5_5: 100%
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
root at test03:~ # pkg info
FreeBSD-casper-12.0.s20190414123806 casper package
FreeBSD-clibs-12.0.s20190414123806 Core C Libraries
FreeBSD-jail-12.0.s20190414123806 Jail Utilities
FreeBSD-lib-12.0.s20190414123806 lib package
FreeBSD-lib80211-12.0.s20190414123806 lib80211 package
...
x265-3.0 ? ? ? ? ? ? ? ? ? ? ? H.265/High Efficiency Video Coding (HEVC) format
xorg-fonts-truetype-7.7_1 ? ? ?X.Org TrueType fonts
xorgproto-2018.4 ? ? ? ? ? ? ? xorg protocol headers
xvid-1.3.5,1 ? ? ? ? ? ? ? ? ? Opensource MPEG-4 codec, based on OpenDivx
yajl-2.1.0 ? ? ? ? ? ? ? ? ? ? Portable JSON parsing and serialization library
in ANSI C
Btw. The missing "revoked" directory problem shoud be fixed with
https://reviews.freebsd.org/D20056[https://reviews.freebsd.org/D20056]
---
So this is a basic jail installation with pkgbase, now to some further
experiments and resulting questions:
?
- i looked into why the userdatabase was missing in the newly created jail.
Well, it is, because the files are not included in the default Freebsd-runtime
package.
? To check if it would work, i built my own Freebsd-runtime package with these
files added. to do this, i copied the runtime.plist file from the $WSTAGEDIR and
modified it.
? - first i removed a lot of duplicated directory entries like
? @dir(root,wheel,0755,) /boot
? @dir(root,wheel,0755,) /boot
? @dir(root,wheel,0755,) /boot
? @dir(root,wheel,0755,) /boot
? ... (and many more)
? - then i added the "missing" files and directory
? @config(root,wheel,0600,) /etc/master.passwd
? @config(root,wheel,0644,) /etc/group
? @dir(root,wheel,0755,) /usr/share/keys/pkg/revoked
??
? - after that i rebuild the Freebsd-runtime package with these changes (found
in /usr/src/Makefile.inc1):
? root at betablock:~ # pkg -o
ABI_FILE=/usr/obj/usr/src/amd64.amd64/worldstage/usr/bin/uname -o
ALLOW_BASE_SHLIBS=yes create -M
/usr/obj/usr/src/amd64.amd64/worldstage/runtime.ucl -p /root/pkgs/runtime.plist
-r /usr/obj/usr/src/amd64.amd64/worldstage --output-dir
/usr/repo/FreeBSD:12:amd64/latest
??
? and rebuilding the repo metadata with:
? root at betablock:~ # pkg repo /usr/repo/FreeBSD:12:amd64/latest
? a jail created with this Freebsd-runtime starts directly, without the steps
mentioned above.
??
? My Question: Is the ommission of these files intentional??
??
? - one last experiment:?i deleted the manpages, the examples, and the /boot
directory (i guessed a jail has no need for it), the jail build from this file
is 75M, the installation of packages like nginx?works like in the original file.
??
? - should the FreeBSD-libcasper package be installed as a dependency of the
FreeBSD-casper package?
?
? - i did not observe more missing libraries like libcasper above, but depending
on the tasks that are?executed or packages that are installed, there might be
more that are missing.
?
So that was my pkgbase-jail experiment, it was fun and i learned some things
about pkgbase. Any comments, tips, tricks and (mild) critique are welcome.
?
Greetings
?
Martin
?