zfs discuss - Aug 2007 - DMU as general purpose transaction engine?

ZFS Experts,

Is it possible to use DMU as general purpose transaction engine? More
specifically, in following order:

1. Create transaction:
tx = dmu_tx_create(os);
error = dmu_tx_assign(tx, TXG_WAIT)

2. Decide what to modify(say create new object):
dmu_tx_hold_bonus(tx, DMU_NEW_OBJECT);
dmu_tx_hold_bonus(tx, dzp->z_id);
dmu_tx_hold_zap(tx, dzp->z_id, TRUE, name);
|
|

3. Commit transaction:
dmu_tx_commit(tx);

The reason I am asking for this particular order because I may not
know the intent of transaction till late in the process.
If it is not possible, can I at least declare that the transaction is
going to change N objects (without specification of each object) and
each change is M blocks at most (without specification of object and
offset). If yes, how?

Thanks,
-Atul

-- 
Atul Vidwansa
Cluster File Systems Inc.
http://www.clusterfs.com

I am not an expert but I think the correct sequence is :

1. dmu_tx_create()
2. dmu_tx_hold_xxxx()
3. dmu_tx_assign()
4. modify the objects as part of the transaction
5. dmu_tx_commit()

see comments in  common/fs/zfs/sys/dmu.h



Thanks
Bhaskar

On Aug 30, 2007  20:16 +1000, Atul Vidwansa wrote:
> Is it possible to use DMU as general purpose transaction engine? More
> specifically, in following order:
> 
> 1. Create transaction:
> tx = dmu_tx_create(os);
> error = dmu_tx_assign(tx, TXG_WAIT)
> 
> 2. Decide what to modify(say create new object):
> dmu_tx_hold_bonus(tx, DMU_NEW_OBJECT);
> dmu_tx_hold_bonus(tx, dzp->z_id);
> dmu_tx_hold_zap(tx, dzp->z_id, TRUE, name);
> |
> |
> 
> 3. Commit transaction:
> dmu_tx_commit(tx);
> 
> The reason I am asking for this particular order because I may not
> know the intent of transaction till late in the process.
> If it is not possible, can I at least declare that the transaction is
> going to change N objects (without specification of each object) and
> each change is M blocks at most (without specification of object and
> offset). If yes, how?
I''m not very familiar with this part of the DMU code, but I''d
suspect
this is not possible.  I suspect reason the dmu_tx_hold_*() functions
need to be called before dmu_tx_assign() is because the DMU is actually
locking those objects, figuring out the maximum number of blocks might
be be modified (based on the objects and parameters), and then
dmu_tx_assign() is actually starting the transaction with that many
blocks.

In ext3, the caller has to work out the number of blocks in the operation
in advance, and generally has to work out the worst possible case.
This is ugly and prone to error (if layouts change all callers must
become aware of them), but at least possible.  With ZFS I think it would
be impossible to do in the caller because of potential changes to snapshots,
the parent checksum updates, etc.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Atul Vidwansa wrote:
> ZFS Experts,
> 
> Is it possible to use DMU as general purpose transaction engine? More
> specifically, in following order:
> 
> 1. Create transaction:
> tx = dmu_tx_create(os);
> error = dmu_tx_assign(tx, TXG_WAIT)
> 
> 2. Decide what to modify(say create new object):
> dmu_tx_hold_bonus(tx, DMU_NEW_OBJECT);
> dmu_tx_hold_bonus(tx, dzp->z_id);
> dmu_tx_hold_zap(tx, dzp->z_id, TRUE, name);
> |
> |
> 
> 3. Commit transaction:
> dmu_tx_commit(tx);
> 
> The reason I am asking for this particular order because I may not
> know the intent of transaction till late in the process.
> If it is not possible, can I at least declare that the transaction is
> going to change N objects (without specification of each object) and
> each change is M blocks at most (without specification of object and
> offset). If yes, how?
You must specify what will be modified (by using dmu_tx_hold_*) before doing 
dmu_tx_assign().  dmu_tx_assign() needs to know what may be modified in order 
to determine if there is enough space available.

You can try to solve this problem by doing dmu_tx_hold_*() on anything that 
*might* be modified.  Hopefully you have enough information up front to 
determine that to some degree of accuracy.  Otherwise, you must go far enough 
in your process to determine what will be modified before doing the 
dmu_tx_hold_*() and dmu_tx_assign().

--matt