freebsd stable - Mar 2013 - Virtio and GEOM labels

I'm running FreeBSD 9-STABLE as a guest under RHEL 6.4 KVM virtualisation. 
I have networking and storage in the FreeBSD guest using the Virtio drivers
(with the virtual disk set to "Virtio" in the definition on the host).
Everything is working nicely: I have a vtnet network adapter and see vtbd
devices for my virtual disks in FreeBSD.  Performance is much better compared
with an emulated IDE device.

The odd thing is that I don't see GEOM labels reflected in /dev.  For
example, I have GPT labels defined in the guest, but I don't see them show
up under /dev/gpt.  Similarly, my UFS labels don't show up under /dev/ufs. 
I *do* see a /dev/gptid.  That appears to be the only label that shows up.

Is there something special I need to do to get GPT and UFS labels to appear when
using Virtio?  It seems to me that Virtio block devices appear to be somewhat
unusual.  Unlike regular ATA and SCSI devices, my vtbd devices don't appear
in the boot dmesg (although a vtblk device does), and "camcontrol
devlist" does not list them.  It's not clear to me how I am supposed to
interact with them other than via basic device I/O through /dev/vtbdX.  I
thought that the virtio_scsi module might make them appear as "da"
devices and able to interacted with via camcontrol, but this doesn't seem to
be the case.

I've included my system dmesg at the end of this message.

Cheers,

Paul.

====Copyright (c) 1992-2013 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
        The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 9.1-STABLE #0 r248465: Mon Mar 18 11:19:23 EDT 2013
    pmather at freebsd9.virtual.lib.vt.edu:/usr/obj/usr/src/sys/KVMTEST amd64
FreeBSD clang version 3.2 (tags/RELEASE_32/final 170710) 20121221
CPU: QEMU Virtual CPU version (cpu64-rhel6) (2100.15-MHz K8-class CPU)
  Origin = "AuthenticAMD"  Id = 0x6d3  Family = 0x6  Model = 0xd 
Stepping = 3
 
Features=0x783f3ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE,SSE2>
  Features2=0x80002001<SSE3,CX16,HV>
  AMD Features=0x22500800<SYSCALL,NX,MMX+,FFXSR,LM>
  AMD Features2=0x73<LAHF,CMP,CR8,ABM,SSE4A>
real memory  = 8589934592 (8192 MB)
avail memory = 8255410176 (7872 MB)
Event timer "LAPIC" quality 400
ACPI APIC Table: <BOCHS  BXPCAPIC>
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 2 package(s) x 1 core(s)
 cpu0 (BSP): APIC ID:  0
 cpu1 (AP): APIC ID:  1
ioapic0 <Version 1.1> irqs 0-23 on motherboard
kbd1 at kbdmux0
acpi0: <BOCHS BXPCRSDT> on motherboard
acpi0: Power Button (fixed)
cpu0: <ACPI CPU> on acpi0
cpu1: <ACPI CPU> on acpi0
atrtc0: <AT realtime clock> port 0x70-0x71,0x72-0x77 irq 8 on acpi0
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <24-bit timer at 3.579545MHz> port 0xb008-0xb00b on acpi0
pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0
pci0: <ACPI PCI bus> on pcib0
pci_link4: Unable to route IRQs: AE_NOT_FOUND
isab0: <PCI-ISA bridge> at device 1.0 on pci0
isa0: <ISA bus> on isab0
atapci0: <Intel PIIX3 WDMA2 controller> port
0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xc000-0xc00f at device 1.1 on pci0
ata0: <ATA channel> at channel 0 on atapci0
ata1: <ATA channel> at channel 1 on atapci0
uhci0: <Intel 82371SB (PIIX3) USB controller> port 0xc020-0xc03f irq 11 at
device 1.2 on pci0
usbus0: controller did not stop
usbus0 on uhci0
pci0: <bridge> at device 1.3 (no driver attached)
vgapci0: <VGA-compatible display> mem
0xf0000000-0xf1ffffff,0xf2000000-0xf2000fff at device 2.0 on pci0
virtio_pci0: <VirtIO PCI Network adapter> port 0xc040-0xc05f mem
0xf2020000-0xf2020fff irq 11 at device 3.0 on pci0
vtnet0: <VirtIO Networking Adapter> on virtio_pci0
virtio_pci0: host features: 0x711fffe3
<EventIdx,RingIndirect,NotifyOnEmpty,RxModeExtra,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxUFO,TxTSOECN,TxTSOv6,TxTSOv4,RxUFO,RxECN,RxTSOv6,RxTSOv4,TxAllGSO,MacAddress,RxChecksum,TxChecksum>
virtio_pci0: negotiated features: 0x110fbba3
<RingIndirect,NotifyOnEmpty,VLanFilter,RxMode,ControlVq,Status,MrgRxBuf,TxTSOECN,TxTSOv6,TxTSOv4,RxECN,RxTSOv6,RxTSOv4,MacAddress,RxChecksum,TxChecksum>
vtnet0: Ethernet address: 52:54:00:3e:48:94
virtio_pci1: <VirtIO PCI Balloon adapter> port 0xc060-0xc07f irq 10 at
device 5.0 on pci0
vtballoon0: <VirtIO Balloon Adapter> on virtio_pci1
virtio_pci1: host features: 0x71000002
<EventIdx,RingIndirect,NotifyOnEmpty,StatsVq>
virtio_pci1: negotiated features: 0x0
virtio_pci2: <VirtIO PCI Block adapter> port 0xc080-0xc0bf mem
0xf2040000-0xf2040fff irq 10 at device 6.0 on pci0
vtblk0: <VirtIO Block Adapter> on virtio_pci2
virtio_pci2: host features: 0x71000654
<EventIdx,RingIndirect,NotifyOnEmpty,Topology,FlushCmd,BlockSize,DiskGeometry,MaxNumSegs>
virtio_pci2: negotiated features: 0x10000254
<RingIndirect,FlushCmd,BlockSize,DiskGeometry,MaxNumSegs>
vtblk0: 8192MB (16777216 512 byte sectors)
atkbdc0: <Keyboard controller (i8042)> port 0x60,0x64 irq 1 on acpi0
atkbd0: <AT Keyboard> irq 1 on atkbdc0
kbd0 at atkbd0
atkbd0: [GIANT-LOCKED]
psm0: <PS/2 Mouse> irq 12 on atkbdc0
psm0: [GIANT-LOCKED]
psm0: model IntelliMouse Explorer, device ID 4
uart0: <Non-standard ns8250 class UART with FIFOs> port 0x3f8-0x3ff irq 4
flags 0x10 on acpi0
orm0: <ISA Option ROM> at iomem 0xc9000-0xc97ff on isa0
sc0: <System console> at flags 0x100 on isa0
sc0: VGA <16 virtual consoles, flags=0x300>
vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0
attimer0: <AT timer> at port 0x40 on isa0
Timecounter "i8254" frequency 1193182 Hz quality 0
Event timer "i8254" frequency 1193182 Hz quality 100
ctl: CAM Target Layer loaded
Timecounters tick every 10.000 msec
usbus0: 12Mbps Full Speed USB v1.0
ugen0.1: <Intel> at usbus0
uhub0: <Intel UHCI root HUB, class 9/0, rev 1.00/1.00, addr 1> on usbus0
cd0 at ata1 bus 0 scbus1 target 0 lun 0
cd0: <QEMU QEMU DVD-ROM 0.12> Removable CD-ROM SCSI-0 device 
cd0: 16.700MB/s transfers (WDMA2, ATAPI 12bytes, PIO 65534bytes)
cd0: Attempt to query device size failed: NOT READY, Medium not present
SMP: AP CPU #1 Launched!
Root mount waiting for: usbus0
uhub0: 2 ports with 2 removable, self powered
Trying to mount root from ufs:/dev/vtbd0p2 [rw]...

On Mar 22, 2013, at 8:14 AM, Paul Mather <paul at gromit.dlib.vt.edu>
wrote:
> I'm running FreeBSD 9-STABLE as a guest under RHEL 6.4 KVM
virtualisation.  I have networking and storage in the FreeBSD guest using the
Virtio drivers (with the virtual disk set to "Virtio" in the
definition on the host).  Everything is working nicely: I have a vtnet network
adapter and see vtbd devices for my virtual disks in FreeBSD.  Performance is
much better compared with an emulated IDE device.
I've had the same experience.
> The odd thing is that I don't see GEOM labels reflected in /dev.  For
example, I have GPT labels defined in the guest, but I don't see them show
up under /dev/gpt.  Similarly, my UFS labels don't show up under /dev/ufs. 
I *do* see a /dev/gptid.  That appears to be the only label that shows up.
I have not encountered this issue. I use virtio block devices and GPT labels
exclusively in multiple FreeBSD 9.1 guests and all mount/function without issue.
How are you referring to your filesystems in /etc/fstab? IIRC GEOM makes
not-in-use labels disappear when a device is in use (e.g. mounted). If you take
a new device, put a labeled GPT partition on it and a labeled UFS partition on
that but don't mount anything, what happens?
> Is there something special I need to do to get GPT and UFS labels to appear
when using Virtio?  It seems to me that Virtio block devices appear to be
somewhat unusual.  Unlike regular ATA and SCSI devices, my vtbd devices
don't appear in the boot dmesg (although a vtblk device does), and
"camcontrol devlist" does not list them.  It's not clear to me how
I am supposed to interact with them other than via basic device I/O through
/dev/vtbdX.  I thought that the virtio_scsi module might make them appear as
"da" devices and able to interacted with via camcontrol, but this
doesn't seem to be the case.
Virtio block devices and virtio SCSI devices are not the same. If you want to
use the virtio_scsi module in FreeBSD you should expose a virtio SCSI device
from the host.

JN