Hi all, This is the patch for IA64 nvram emulation. Note that Nvram Option in configure file is also prerequisite when creating a Vti domain. -----Original Message----- From: Alex Williamson [mailto:alex.williamson@hp.com] Sent: 2006年10月16日 5:18 To: Huang, Xinmei Cc: xen-ia64-devel@lists.xensource.com Subject: Re: [Xen-ia64-devel][PATCH]Nvram patch On Fri, 2006-10-13 at 14:37 +0800, Huang, Xinmei wrote:> Hi all, > Nvram emulation solution is that: Qemu uses one file to emulate nvram. > Qemu would load/save relevant content from/to nvram file when GFW sends > command through I/O port. > Nvram option is added in config file to specify a per-domain nvram file > path, which can''t be shared by different domains. Nvram file would be > created if non-existent. > A new GFW which supports nvram emulation is prerequisite.This looks good, but it''s mainly adding new functionality to common code. Other architectures may wish to leverage this work too. I think it should therefore be submitted to xen-devel. Thanks, Alex -- Alex Williamson HP Open Source & Linux Org. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Christian, Do you have comments on this patch? NVRAM is a useful feature for IA64 VT-I domain, especially for windows 2003 booting. Best Regards Ke Huang, Xinmei <> write on 2006-10-18 9:40:> Hi all, > This is the patch for IA64 nvram emulation. > Note that Nvram Option in configure file is also prerequisite when > creating a Vti domain. -----Original Message----- > From: Alex Williamson [mailto:alex.williamson@hp.com] > Sent: 2006年10月16日 5:18 > To: Huang, Xinmei > Cc: xen-ia64-devel@lists.xensource.com > Subject: Re: [Xen-ia64-devel][PATCH]Nvram patch > > On Fri, 2006-10-13 at 14:37 +0800, Huang, Xinmei wrote: >> Hi all, >> Nvram emulation solution is that: Qemu uses one file to emulate >> nvram. Qemu would load/save relevant content from/to nvram file when >> GFW sends command through I/O port. Nvram option is added in config >> file to specify a per-domain nvram file path, which can''t be shared >> by different domains. Nvram file would be created if non-existent. A >> new GFW which supports nvram emulation is prerequisite. > > This looks good, but it''s mainly adding new functionality to common > code. Other architectures may wish to leverage this work too. I > think > it should therefore be submitted to xen-devel. Thanks, > > Alex_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Do you have comments on this patch? NVRAM is a useful feature for IA64 VT-I > domain, especially for windows 2003 booting.What does it use the NVRAM for? Is the meaning common across all OSes that use it? How many bytes are used? Does the data actually need to be changed, or would initializing it to a constant work? If the data is static, I''m wandering whether it would be better to store this data explicitly as part of the domain config rather than referencing another file. Thanks, Ian> Best Regards > Ke > > Huang, Xinmei <> write on 2006-10-18 9:40: > > Hi all, > > This is the patch for IA64 nvram emulation. > > Note that Nvram Option in configure file is also prerequisite when > > creating a Vti domain. -----Original Message----- > > From: Alex Williamson [mailto:alex.williamson@hp.com] > > Sent: 2006年10月16日 5:18 > > To: Huang, Xinmei > > Cc: xen-ia64-devel@lists.xensource.com > > Subject: Re: [Xen-ia64-devel][PATCH]Nvram patch > > > > On Fri, 2006-10-13 at 14:37 +0800, Huang, Xinmei wrote: > >> Hi all, > >> Nvram emulation solution is that: Qemu uses one file to emulate > >> nvram. Qemu would load/save relevant content from/to nvram file when > >> GFW sends command through I/O port. Nvram option is added in config > >> file to specify a per-domain nvram file path, which can''t be shared > >> by different domains. Nvram file would be created if non-existent. A > >> new GFW which supports nvram emulation is prerequisite. > > > > This looks good, but it''s mainly adding new functionality to common > > code. Other architectures may wish to leverage this work too. I > > think > > it should therefore be submitted to xen-devel. Thanks, > > > > Alex > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006年10月30日 17:58:>> Do you have comments on this patch? NVRAM is a useful feature for >> IA64 VT-I domain, especially for windows 2003 booting. > > What does it use the NVRAM for? Is the meaning common across all OSes > that use it? How many bytes are used? Does the data actually need to > be changed, or would initializing it to a constant work?NVRAM is used by IA64 guest firmware (EFI) to store non-volatile variable. EFI variable is a concept similar to the linux bash environment variable. EFI and OS (Linux, Windows 2003, Windows Vista) use variable to communicate information. They all use EFI interface GetVariable()/SetVariable() to handle variable. One usage example is: to boot Windows 2003, we need run an efi tool "nvrboot" to import boot option into one variable, then EFI boot manager can read this variable and boot windows 2003. Without NVRAM support, we must run nvrboot everytime creating VT-I windows 2003 domain. with NVRAM support, nvrboot only need to run once. This is especially useful for nightly test. The size of NVRAM is implementation specific. EFI spec does not define that. In current implementation, it is 64KB. The data is dynamically changed, per EFI or guest OS SetVariable() request.> > If the data is static, I''m wandering whether it would be better to > store this data explicitly as part of the domain config rather than > referencing another file. > > Thanks, > Ian >The data is dynamic, so I am afraid it is hard to store NVRAM data in domain config. Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30/10/06 14:05, "Yu, Ke" <ke.yu@intel.com> wrote:> The data is dynamic, so I am afraid it is hard to store NVRAM data in domain > config.There must be a possible static good configuration to allow you to boot Oses you care about? Do changes happen during guest execution that you really care about preserving across reboots? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > The data is dynamic, so I am afraid it is hard to store > NVRAM data in > > domain config. > > There must be a possible static good configuration to allow > you to boot Oses you care about? Do changes happen during > guest execution that you really care about preserving across reboots?Further, storing it in a file will create compilcations when we move to running qemu in stub domains -- we''ll need a way of passing it across xenbus. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006?10?30? 22:23:>>> The data is dynamic, so I am afraid it is hard to store NVRAM data >>> in domain config. >> >> There must be a possible static good configuration to allow >> you to boot Oses you care about? Do changes happen during >> guest execution that you really care about preserving across reboots?Yes, static good configurations can be made for different guest OS (RHEL, SLES, Win 2003, Win Vista). But I am afraid this workaround will have potential issue, because we are unable to predict how guest OS would use variable.> > Further, storing it in a file will create compilcations when we move > to running qemu in stub domains -- we''ll need a way of passing it > across xenbus.Could you please elaborate what the "complications" is? Per my understanding, even without NVRAM file, qemu in stub domain still need to read/write the disk image file. It is also OK to store NVRAM in xenstore, but seems xenstore have no capability to store binary data, it can only store null-termincated strings. If we want to directly store EFI variable in xenstore instead of sotre NVRAM binary, we need to para-virtualize guest firmware GetVariable/Setvariable interface, which is more complicated.> > > Ian_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Ian Pratt write on 2006?10?30? 22:23: > >>> The data is dynamic, so I am afraid it is hard to store > NVRAM data > >>> in domain config. > >> > >> There must be a possible static good configuration to allow you to > >> boot Oses you care about? Do changes happen during guest execution > >> that you really care about preserving across reboots? > > Yes, static good configurations can be made for different > guest OS (RHEL, SLES, Win 2003, Win Vista). But I am afraid > this workaround will have potential issue, because we are > unable to predict how guest OS would use variable.Is the variable contents the same for all installs of each of the OSes you list?> > Further, storing it in a file will create compilcations > when we move > > to running qemu in stub domains -- we''ll need a way of passing it > > across xenbus. > > Could you please elaborate what the "complications" is? Per > my understanding, even without NVRAM file, qemu in stub > domain still need to read/write the disk image file.You won''t be able to write stuff to a file directly, you''ll need to use a front/back driver, which is needlessly complicated. xenbus is definitely the way to go.> It is also OK to store NVRAM in xenstore, but seems xenstore > have no capability to store binary data, it can only store > null-termincated strings. If we want to directly store EFI > variable in xenstore instead of sotre NVRAM binary, we need > to para-virtualize guest firmware GetVariable/Setvariable > interface, which is more complicated.You''ll have to escape the NULLs. It might be easiest just to store the hex string. You don''t need to paravirtulize it as qemu can do the trivial conversion. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006?10?31? 19:46:>> Ian Pratt write on 2006?10?30? 22:23: >>>>> The data is dynamic, so I am afraid it is hard to store NVRAM data >>>>> in domain config. >>>> >>>> There must be a possible static good configuration to allow you to >>>> boot Oses you care about? Do changes happen during guest execution >>>> that you really care about preserving across reboots? >> >> Yes, static good configurations can be made for different >> guest OS (RHEL, SLES, Win 2003, Win Vista). But I am afraid >> this workaround will have potential issue, because we are >> unable to predict how guest OS would use variable. > > Is the variable contents the same for all installs of each of the OSes > you list?No, different OS has differrent configurations.> >>> Further, storing it in a file will create compilcations when we move >>> to running qemu in stub domains -- we''ll need a way of passing it >>> across xenbus. >> >> Could you please elaborate what the "complications" is? Per >> my understanding, even without NVRAM file, qemu in stub >> domain still need to read/write the disk image file. > > You won''t be able to write stuff to a file directly, you''ll need to > use a front/back driver, which is needlessly complicated. xenbus is > definitely the way to go. > >> It is also OK to store NVRAM in xenstore, but seems xenstore >> have no capability to store binary data, it can only store >> null-termincated strings. If we want to directly store EFI >> variable in xenstore instead of sotre NVRAM binary, we need >> to para-virtualize guest firmware GetVariable/Setvariable >> interface, which is more complicated. > > You''ll have to escape the NULLs. It might be easiest just to store the > hex string. > > You don''t need to paravirtulize it as qemu can do the trivial > conversion. > > IanOK. this should work. Then the only concern is the size of NVRAM. 64KB data is quite large for xenstore. Is it acceptable to xenstore? Maybe qemu need to compress the data first. Usually, most of the NVRAM content is free area, which is continuous byte "0xFF", so compresion can reduce the size significantly. Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> >> Yes, static good configurations can be made for different guest OS > >> (RHEL, SLES, Win 2003, Win Vista). But I am afraid this workaround > >> will have potential issue, because we are unable to > predict how guest > >> OS would use variable. > > > > Is the variable contents the same for all installs of each > of the OSes > > you list? > > No, different OS has differrent configurations.But do all installs of the same OS have the same variable contents? That was the question.> >>> Further, storing it in a file will create compilcations > when we move > >>> to running qemu in stub domains -- we''ll need a way of passing it > >>> across xenbus. > >> > >> Could you please elaborate what the "complications" is? Per my > >> understanding, even without NVRAM file, qemu in stub domain still > >> need to read/write the disk image file. > > > > You won''t be able to write stuff to a file directly, you''ll need to > > use a front/back driver, which is needlessly complicated. xenbus is > > definitely the way to go. > > > >> It is also OK to store NVRAM in xenstore, but seems > xenstore have no > >> capability to store binary data, it can only store > null-termincated > >> strings. If we want to directly store EFI variable in xenstore > >> instead of sotre NVRAM binary, we need to para-virtualize guest > >> firmware GetVariable/Setvariable interface, which is more > >> complicated. > > > > You''ll have to escape the NULLs. It might be easiest just > to store the > > hex string. > > > > You don''t need to paravirtulize it as qemu can do the trivial > > conversion. > > > > Ian > > OK. this should work. Then the only concern is the size of > NVRAM. 64KB data is quite large for xenstore. Is it > acceptable to xenstore? Maybe qemu need to compress the data > first. Usually, most of the NVRAM content is free area, which > is continuous byte "0xFF", so compresion can reduce the size > significantly.Break it into 64 byte (128 character) chunks and only populate nodes that are not all ones. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006?10?31? 22:45:>>>> Yes, static good configurations can be made for different guest OS >>>> (RHEL, SLES, Win 2003, Win Vista). But I am afraid this workaround >>>> will have potential issue, because we are unable to predict how >>>> guest OS would use variable. >>> >>> Is the variable contents the same for all installs of each of the >>> OSes you list? >> >> No, different OS has differrent configurations. > > But do all installs of the same OS have the same variable contents? > That was the question.Oh, I misunderstand your question. Yes. The same OS installation have same variable contents.> >>>>> Further, storing it in a file will create compilcations when we >>>>> move to running qemu in stub domains -- we''ll need a way of >>>>> passing it across xenbus. >>>> >>>> Could you please elaborate what the "complications" is? Per my >>>> understanding, even without NVRAM file, qemu in stub domain still >>>> need to read/write the disk image file. >>> >>> You won''t be able to write stuff to a file directly, you''ll need to >>> use a front/back driver, which is needlessly complicated. xenbus is >>> definitely the way to go. >>> >>>> It is also OK to store NVRAM in xenstore, but seems xenstore have >>>> no capability to store binary data, it can only store >>>> null-termincated strings. If we want to directly store EFI >>>> variable in xenstore instead of sotre NVRAM binary, we need to >>>> para-virtualize guest firmware GetVariable/Setvariable interface, >>>> which is more complicated. >>> >>> You''ll have to escape the NULLs. It might be easiest just to store >>> the hex string. >>> >>> You don''t need to paravirtulize it as qemu can do the trivial >>> conversion. >>> >>> Ian >> >> OK. this should work. Then the only concern is the size of >> NVRAM. 64KB data is quite large for xenstore. Is it >> acceptable to xenstore? Maybe qemu need to compress the data >> first. Usually, most of the NVRAM content is free area, which >> is continuous byte "0xFF", so compresion can reduce the size >> significantly. > > Break it into 64 byte (128 character) chunks and only populate nodes > that are not all ones. > > IanOK. then the xensotor content will looks like this: nvram = " " # nvram dir vti-domain1 = " " # domain name 0 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, data offset = 0*64 1 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, data offset = 1*64 ......... # skip data block of all 0xff vti-domain2 = " " .......... Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > But do all installs of the same OS have the same variable contents? > > That was the question. > > Oh, I misunderstand your question. Yes. The same OS installation havesame> variable contents.In which case, wouldn''t it just be a whole lot simpler to have qemu-dm know about these constants and just have a configuration parameter to cause it to serve up the right one when asked? We probably should have a ''guest_type'' field in the guest config anyway as we''ll likely want to use this for enabling/tuning various optimizations. I know this isn''t ideal from a purist virtualization point of view, but it seems like a practical solution.> OK. then the xensotor content will looks like this: > nvram = " " # nvram dir > vti-domain1 = " " # domain name > 0 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, dataoffset > 0*64> 1 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, dataoffset > 1*64> ......... # skip datablock> of all 0xff > vti-domain2 = " " > ..........We''ll have to think about how this data gets persisted (I''d trigger a watcher and store it outside of xenstore). Look at the xen-api.hg tree to see how stuff gets persisted. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006?10?31? 23:33:> > In which case, wouldn''t it just be a whole lot simpler to have qemu-dm > know about these constants and just have a configuration parameter to > cause it to serve up the right one when asked? > > We probably should have a ''guest_type'' field in the guest config > anyway as we''ll likely want to use this for enabling/tuning various > optimizations. > > I know this isn''t ideal from a purist virtualization point of view, > but it seems like a practical solution.Since we know the final solution will be storing the nvram through xenbus, and this is not complex, I''d prefer to directly work out the final solution.> >> OK. then the xensotor content will looks like this: >> nvram = " " # nvram dir >> vti-domain1 = " " # domain name >> 0 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, data >> offset = 0*64 1 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 >> characters, data offset = 1*64 ......... >> # skip data block of all 0xff vti-domain2 = " " >> .......... > > We''ll have to think about how this data gets persisted (I''d trigger a > watcher and store it outside of xenstore). Look at the xen-api.hg tree > to see how stuff gets persisted. > > IanI am not familiar with the xan-api project, so could you please give more info on "trigger a watcher and store it outside of xenstore"? Does xenstore has the interface to write data outside of xenstore. Or do you mean having a separate thread to watch, and the thread will write data outside of xenstore. In this way, qemu should have ringbuffer to pass the data to the thread. Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> >> OK. then the xensotor content will looks like this: > >> nvram = " " # nvram dir > >> vti-domain1 = " " # domain name > >> 0 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 characters, data > >> offset = 0*64 1 = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxx" # 128 > >> characters, data offset = 1*64 ......... > >> # skip data block of all 0xff vti-domain2 = " " > >> .......... > > > > We''ll have to think about how this data gets persisted (I''d triggera> > watcher and store it outside of xenstore). Look at the xen-api.hgtree> > to see how stuff gets persisted. > I am not familiar with the xan-api project, so could you please givemore> info on "trigger a watcher and store it outside of xenstore"?See xenbits.xensource.com/ext/xen-api.hg Various parts of a domain''s runtime config are persisted in a database, and the nvram state could be added to this. There''s no good example to copy as the nvram state is unusual in that its written by both qemu rather than the control tools. Xenbus watchers let you do this, though.> Does xenstore > has the interface to write data outside of xenstore.You can register a watcher to be told when someone writes to a section of the store then arrange to persist the data. You can add this to xend.> Or do you mean having > a separate thread to watch, and the thread will write data outside of > xenstore. In this way, qemu should have ringbuffer to pass the data tothe> thread.No extra ring buffer is needed or wanted. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt write on 2006?11?1? 16:25:>> I am not familiar with the xan-api project, so could you please give >> more info on "trigger a watcher and store it outside of xenstore"? > > See xenbits.xensource.com/ext/xen-api.hg > > Various parts of a domain''s runtime config are persisted in a > database, and the nvram state could be added to this. There''s no good > example to copy as the nvram state is unusual in that its written by > both qemu rather than the control tools. Xenbus watchers let you do > this, though. > >> Does xenstore >> has the interface to write data outside of xenstore. > > You can register a watcher to be told when someone writes to a section > of the store then arrange to persist the data. You can add this to > xend. > >> Or do you mean having >> a separate thread to watch, and the thread will write data outside of >> xenstore. In this way, qemu should have ringbuffer to pass the data >> to the thread. > > No extra ring buffer is needed or wanted. > > IanThank you for the info. It is clear to me now. we will update the patch as following: xenstore: local= " " domain = "" x = "" # domain id nvram # nvram dir file # nvram file path data_in # data index + 64 bytes data, 128 characters, for qemu write data_out # data index + 64 bytes data, 128 characters, for qemu read xend register two watches on data_in node and data_out node. * qemu write data: qemu write "index+data" to data_in node, xend data_in watch handler will copy the data to persisted file. * qemu read data: qemu write index to data_out node; xend data_out watch handler copy data from persisted file to data_out node; qemu read data_out node. Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Thank you for the info. It is clear to me now. we will update > the patch as following: > > xenstore: > local= " " > domain = "" > x = "" # domain id > nvram # nvram dir > file # nvram file path > data_in # data index + 64 bytes data, > 128 characters, for qemu write > data_out # data index + 64 bytes data, 128 > characters, for qemu read > > xend register two watches on data_in node and data_out node. > * qemu write data: qemu write "index+data" to data_in node, > xend data_in watch handler will copy the data to persisted file. > * qemu read data: qemu write index to data_out node; xend > data_out watch handler copy data from persisted file to > data_out node; qemu read data_out node.I don''t think you need the file path -- the data should be stored in the xend ''database'' until you come to export the VM. After initialisation, all the writes to the data will come from qemu, hence I don''t see why you need data_in and data_out. Can''t you just have a ''data'' node? Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
2006/11/1, Ian Pratt <m+Ian.Pratt@cl.cam.ac.uk>:> > I don''t think you need the file path -- the data should be stored in the > xend ''database'' until you come to export the VM.Maybe I misunderstand something. the only database I see is the xenstore database, in which we don''t want to save nvram data. is there another database?> > After initialisation, all the writes to the data will come from qemu, > hence I don''t see why you need data_in and data_out. Can''t you just have > a ''data'' node? > > Ian >The reason of two node is that nvram is a bit special, since it need to communicate with external data file, not just xenstore data. To be more detail, the qemu need to read and write the data. Writing is simple: just write to data_in node and trigger watch handler. read is a little bit complex: qemu write index to data_out first, then watch handler feed data into data_out, finally qemu read the data_out node. if there is xend database, maybe only one node is needed. need more info from you. Best Regards Ke _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel