Xen devel - Oct 2008 - The memory type of non-RAM address is WB by default?

In changeset 17471:cd5dc735bdf3(x86, hvm: Lots of MTRR/PAT emulation cleanup),
the default memory type is set to WB and the type of each special memory range
is set properly. Specially, [3.75G, 4G) is assumed as PCI space and marked as
UC.

I meet with an issue:
When I "xm pci-attach" a NIC into RHEL5.1 Linux guest(memory=256M),
the virtual MMIO BAR assigned by guest starts from 0x20000000 and I
don''t find guest kernel tries to set the MTRR registers at all; as a
result, the type of the pfn is WB, but in hypervisor, the host mfn is UC, so
when guest accesses the vBAR, there are many lines of such warnings printed:

(XEN) mtrr.c:400:d10 Conflict occurs for a given guest l1e flags:163 at 20000000
(the effective mm type:6), because the host mtrr type is:0

Here I think the device should work normally since the actual host mfn  is UC,
but we might as well fix such inconsistency.
Can''t we set the default type to UC?

BTW: when I statically assign a NIC to guest via config file, since the vBAR
assigned by hvmloader starts from 0xF0000000, I don''t see the warnings.

Thanks,
-- Dexuan




_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Why take any notice of guest MTRR/PAT in cases where the host knows better?

 -- Keir

On 23/10/08 10:36, "Cui, Dexuan" <dexuan.cui@intel.com> wrote:
> In changeset 17471:cd5dc735bdf3(x86, hvm: Lots of MTRR/PAT emulation
cleanup),
> the default memory type is set to WB and the type of each special memory
range
> is set properly. Specially, [3.75G, 4G) is assumed as PCI space and marked
as
> UC.
> 
> I meet with an issue:
> When I "xm pci-attach" a NIC into RHEL5.1 Linux
guest(memory=256M), the
> virtual MMIO BAR assigned by guest starts from 0x20000000 and I
don''t find
> guest kernel tries to set the MTRR registers at all; as a result, the type
of
> the pfn is WB, but in hypervisor, the host mfn is UC, so when guest
accesses
> the vBAR, there are many lines of such warnings printed:
> 
> (XEN) mtrr.c:400:d10 Conflict occurs for a given guest l1e flags:163 at
> 20000000 (the effective mm type:6), because the host mtrr type is:0
> 
> Here I think the device should work normally since the actual host mfn  is
UC,
> but we might as well fix such inconsistency.
> Can''t we set the default type to UC?
> 
> BTW: when I statically assign a NIC to guest via config file, since the
vBAR
> assigned by hvmloader starts from 0xF0000000, I don''t see the
warnings.
> 
> Thanks,
> -- Dexuan
> 
> 
> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

The issue with setting the WB/UC boundary at top of RAM rather than at 3.75G
is that this boundary is general no more than page-aligned. Hence you''d
need
tens of variable MTRR ranges to express it; considerably more than the eight
which we supply. This is because of the requirement for each such range to
be power-of-two sized and aligned.

 -- Keir

On 23/10/08 10:49, "Keir Fraser" <keir.fraser@eu.citrix.com>
wrote:
> Why take any notice of guest MTRR/PAT in cases where the host knows better?
> 
>  -- Keir
> 
> On 23/10/08 10:36, "Cui, Dexuan" <dexuan.cui@intel.com>
wrote:
> 
>> In changeset 17471:cd5dc735bdf3(x86, hvm: Lots of MTRR/PAT emulation
>> cleanup), 
>> the default memory type is set to WB and the type of each special
memory
>> range 
>> is set properly. Specially, [3.75G, 4G) is assumed as PCI space and
marked as
>> UC.
>> 
>> I meet with an issue:
>> When I "xm pci-attach" a NIC into RHEL5.1 Linux
guest(memory=256M), the
>> virtual MMIO BAR assigned by guest starts from 0x20000000 and I
don''t find
>> guest kernel tries to set the MTRR registers at all; as a result, the
type of
>> the pfn is WB, but in hypervisor, the host mfn is UC, so when guest
accesses
>> the vBAR, there are many lines of such warnings printed:
>> 
>> (XEN) mtrr.c:400:d10 Conflict occurs for a given guest l1e flags:163 at
>> 20000000 (the effective mm type:6), because the host mtrr type is:0
>> 
>> Here I think the device should work normally since the actual host mfn 
is
>> UC, 
>> but we might as well fix such inconsistency.
>> Can''t we set the default type to UC?
>> 
>> BTW: when I statically assign a NIC to guest via config file, since the
vBAR
>> assigned by hvmloader starts from 0xF0000000, I don''t see the
warnings.
>> 
>> Thanks,
>> -- Dexuan
>> 
>> 
>> 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> Why take any notice of guest MTRR/PAT in cases where the host knows
> better?
>
So we don''t treat this warning as a bug and would not fix it. Correct?
I don''t like warning. :-)

Thanks,
-- Dexuan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> The issue with setting the WB/UC boundary at top of RAM rather than
> at 3.75G is that this boundary is general no more than page-aligned.
> Hence you''d need tens of variable MTRR ranges to express it;
> considerably more than the eight which we supply. This is because of
> the requirement for each such range to be power-of-two sized and
> aligned.
Thanks for the explanation. I understand the situation.

Thanks,
-- Dexuan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 23/10/08 11:01, "Cui, Dexuan" <dexuan.cui@intel.com> wrote:
> Keir Fraser wrote:
>> Why take any notice of guest MTRR/PAT in cases where the host knows
>> better?
>> 
> So we don''t treat this warning as a bug and would not fix it.
Correct?
> I don''t like warning. :-)
Well, one approach would be not to warn! :-)

Seriously: for anything which is not guest RAM, the host should really know
best what memory type it should be mapped as (probably UC or WC). Hence why
mess around validating a guest-requested memory type? The situation is
different of course for guest RAM being used in non-WB contexts for device
communications: in that case the guest does know best.

The alternative is to give guests a shedload more MTRR ranges. This is kind
of clunky and will also slow down the MTRR-range search which Xen currently
does when working out guest effective memory type. But it does have the
advantage of letting us have default type as UC, which I guess is the
default type on just about every real machine in the world?

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> Well, one approach would be not to warn! :-)
>
> Seriously: for anything which is not guest RAM, the host should
> really know best what memory type it should be mapped as (probably UC
> or WC). Hence why mess around validating a guest-requested memory
> type?
This method looks good. I''ll try to make a patch.
> The alternative is to give guests a shedload more MTRR ranges. This
> is kind of clunky and will also slow down the MTRR-range search which
> Xen currently does when working out guest effective memory type.  But
>  it does have the advantage of letting us have default type as UC,
Maybe this method looks not so appealing. :-)
> I guess is the default type on just about every real machine in
> the world?
I''m not sure, maybe it''s platform-dependent?

Thanks,
-- Dexuan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 23/10/08 11:24, "Cui, Dexuan" <dexuan.cui@intel.com> wrote:
> Keir Fraser wrote:
>> Well, one approach would be not to warn! :-)
>> 
>> Seriously: for anything which is not guest RAM, the host should
>> really know best what memory type it should be mapped as (probably UC
>> or WC). Hence why mess around validating a guest-requested memory
>> type?
> This method looks good. I''ll try to make a patch.
Thanks. I agree this is probably the best approach.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir/Dexuan,
	Attached the patch, which only checks the conflict in RAM range.
Keir Fraser wrote:
> On 23/10/08 11:24, "Cui, Dexuan" <dexuan.cui@intel.com>
wrote:
> 
>> Keir Fraser wrote:
>>> Well, one approach would be not to warn! :-)
>>> 
>>> Seriously: for anything which is not guest RAM, the host should
>>> really know best what memory type it should be mapped as (probably
>>> UC or WC). Hence why mess around validating a guest-requested
memory
>>> type?
>> This method looks good. I''ll try to make a patch.
> 
> Thanks. I agree this is probably the best approach.
> 
>  -- Keir


Best Regards,
Disheng, Su

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Keir,
	Please apply the attached patch again...
	p2m_is_ram is better than mfn_valid:)
Su, Disheng wrote:
> Hi Keir/Dexuan,
> 	Attached the patch, which only checks the conflict in RAM range.
> Keir Fraser wrote:
>> On 23/10/08 11:24, "Cui, Dexuan" <dexuan.cui@intel.com>
wrote:
>> 
>>> Keir Fraser wrote:
>>>> Well, one approach would be not to warn! :-)
>>>> 
>>>> Seriously: for anything which is not guest RAM, the host should
>>>> really know best what memory type it should be mapped as
(probably
>>>> UC or WC). Hence why mess around validating a guest-requested
>>>> memory type?
>>> This method looks good. I''ll try to make a patch.
>> 
>> Thanks. I agree this is probably the best approach.
>> 
>>  -- Keir
> 
> 
> 
> Best Regards,
> Disheng, Su


Best Regards,
Disheng, Su

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel