crossbow discuss - Mar 2008 - Brussels persistence proposal review

Here is the first draft of the Brussels persistence fasttrack proposal, 
please review. (Grammar/style suggestions are also welcome, off the list).

I am also looking for a PSARC sponsor. I could sponsor myself, but 
that''s less fun.

thanks,
-Artem


4.1. Summary

PSARC/2007/429 introduced network driver properties that persist across
system reboot, but not across link plumb/unplumb. This case seeks to
fix that deficiency.

Property persistence across reboots is currently achieved by having
svc:/network/physical invoke "dladm init-linkprop" on startup, which
causes all available links to plumb and properties pushed into
respective drivers. Persistence across plumb/unplumb can be
achieved manually using dladm:

	ifconfig <link> plumb
	dladm init-linkprop <link>

We propose that this happens automatically each time a link is plumbed.


4.2. Details

The kernel MAC layer pushes properties into drivers via mc_setprop
callback. From the driver standpoint, link plumbing and unplumbing
correspond, effectively, to mc_start and mc_stop callbacks. We propose
that for each persistent property mc_setprop is invoked prior to
mc_start, but after attach(9E) and mac_register() are completed.
Drivers can save property values in their soft state and configure
hardware accordingly in mc_start.

Implementation utilizes the dlmgmtd daemon (PSARC/2006/499). A door
upcall is invoked from the MAC layer, causing dlmgmtd to initiate the
equivalent of "dladm init-linkprop <link>" using libdladm. The
door call
returns upon operation completion.

     Note: The project team considered reading persistent store
     (currently /etc/dladm/dataconf.conf) directly from the kernel,
     similar to the driver.conf mechanism. But this seems future unproof,
     as the repository might eventually move to SMF, and there is no SMF
     access in the kernel.

Although the above should be sufficient, in practice, the way today''s
drivers are coded, some of them might require properties before
attach(9E) completes. To make life easier for driver developers,
we propose to add "pull" style property retrieval, in addition to
"push" style mc_setprop callback:


int mac_prop_init(const char *link, mac_prop_handle_t *mph)

Returns a handle to be used later with mac_prop_get(). For the
''link''
argument, physical NIC drivers, which do not have access to vanity
name services, should provide the device instance name, e.g. "bge0".


void mac_prop_fini(mac_prop_handle_t mph)

Releases the handle.


int mac_prop_get(mac_prop_handle_t mph, const char *name, void *value,
     uint_t size, uint_t *rsizep)

If the requested property exists in the persistent store, up to
''size''
bytes are copied into ''value''. If buffer is too small, real
size is
returned in ''rsizep'' and the function returns EAGAIN. If
property
does not exist in the persistent store, ENOENT is returned.

These functions do not require links to be in any particular state, so
can be called from attach(9E) or any other kernel context. They could
also be potentially useful for virtual links in the future.


4.3 Even more details

The following interfaces are less interesting in terms of architecture,
but are worth mentioning.

Door upcall interface:

command:      DLMGMT_CMD_LINKPROP_INIT
argument:     dlmgmt_door_linkprop_init_t
flag:         DLMGMT_LINKPROP_MAC_ONLY
return value: dlmgmt_linkprop_init_retval_t


Additional flags for dld_ioc_prop_t:

DLD_PROP_FLAG_MAC_ONLY

This flags indicates to the MAC layer that the property passed from
userland via DLDIOCSETPROP is intended for MAC only and should not be
pushed into drivers via mc_setprop. Used in mac_prop_init()
implementation.

DLD_PROP_FLAG_MAC_NAME

This flag indicates that dld should use the provided MAC name
instead of link ID. Before a device is fully attached, using
vanity name services for mapping can be problematic.

Additional entry in dld_ioc_prop_t:

char    pr_macname[LIFNAMSIZ];

Used instead of the pr_linkid when DLD_PROP_FLAG_MAC_NAME is set.


4.4. Interfaces

Interface                     | Stability
------------------------------+-----------------------
mac_prop_handle_t             | Consolidation Private
mac_prop_init                 | Consolidation Private
mac_prop_fini                 | Consolidation Private
mac_prop_get                  | Consolidation Private
DLMGMT_CMD_LINKPROP_INIT      | Consolidation Private
dlmgmt_door_linkprop_init_t   | Consolidation Private
dlmgmt_linkprop_init_retval_t | Consolidation Private
DLMGMT_LINKPROP_MAC_ONLY      | Consolidation Private
DLD_PROP_FLAG_MAC_ONLY        | Consolidation Private
DLD_PROP_FLAG_MAC_NAME        | Consolidation Private
------------------------------+-----------------------

Release binding: minor


4.5. References

PSARC/2007/429 Brussels - enhanced network driver configuration via dladm
PSARC/2006/499 Clearview Nemo unification and vanity naming

Artem Kachitchkine writes:
> I am also looking for a PSARC sponsor. I could sponsor myself, but 
> that''s less fun.
It''s far less work for everyone else, and it''s likely that you
know
your proposal best and are thus the best possible sponsor.  I''d
recommend it.

(If you''re worried about "conflict of interest" or some such,
don''t.
There''s no such issue here.)

As for the proposal itself, it looks good.  I particularly like that
you''re not relying on ifconfig itself and are instead making it
functional for all possible link users.

-- 
James Carlson, Solaris Networking              <james.d.carlson at
sun.com>
Sun Microsystems / 35 Network Drive        71.232W   Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757   42.496N   Fax +1 781 442 1677

> It''s far less work for everyone else, and it''s likely
that you know
> your proposal best and are thus the best possible sponsor.  I''d
> recommend it.
OK, makes sense. The less work the better.

-Artem

>> int mac_prop_init(const char *link, mac_prop_handle_t *mph)
>>
>> Returns a handle to be used later with mac_prop_get(). For the
''link''
>> argument, physical NIC drivers, which do not have access to vanity
>> name services, should provide the device instance name, e.g.
"bge0".
> 
> Physical link drivers do not necessarily "know" that name. They
know the
> instance number, and driver name. I''d rather see an alternative
wrapper
> version for NIC drivers that takes the dev_info_t, and an
"instance"
> number (ala the mac_register_t) instead of a link name. I think it would 
> be architecturally cleaner to use consistent link identification for 
> this as well as mac_register.
Yeah, I guess passing dev_info_t would be cleaner and consistent with 
mac_register(). And if a "smarter" consumer comes along, it would be 
trivial to add another wrapper that takes a linkname.

-Artem