Xen devel - Jun 2013 - [PATCH] xen/arm: Set up Versatile Express timer frequency to 24 Mhz

On TC2, the timer frequency was set to 100 Mhz which slows down the whole
platform.
When Linux is running on bare metal, the frequency is 24 Mhz.

"sleep 60" in dom0 takes:
  - 4 mins with a frequency equals to 100 Mhz
  - 1 min with a frequency equals to 24 Mhz

Signed-off-by: Julien Grall <julien.grall@linaro.org>
---
 xen/include/asm-arm/platforms/vexpress.h |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/include/asm-arm/platforms/vexpress.h
b/xen/include/asm-arm/platforms/vexpress.h
index 982a293..014509b 100644
--- a/xen/include/asm-arm/platforms/vexpress.h
+++ b/xen/include/asm-arm/platforms/vexpress.h
@@ -39,7 +39,7 @@ int vexpress_syscfg(int write, int function, int device,
uint32_t *data);
 #define V2M_GIC_BASE_ADDRESS        0x2c000000
 
 /* Timer''s frequency */
-#define V2M_TIMER_FREQUENCY         0x5f5e100 /* 100 Mhz */
+#define V2M_TIMER_FREQUENCY         (24 * 1000 * 1000) /* 24 Mhz */
 
 #endif /* __ASSEMBLY__ */
 
-- 
1.7.10.4

At 14:10 +0100 on 24 Jun (1372083058), Julien Grall
wrote:
> On TC2, the timer frequency was set to 100 Mhz which slows down the whole
> platform.
> When Linux is running on bare metal, the frequency is 24 Mhz.
> 
> "sleep 60" in dom0 takes:
>   - 4 mins with a frequency equals to 100 Mhz
>   - 1 min with a frequency equals to 24 Mhz
> 
> Signed-off-by: Julien Grall <julien.grall@linaro.org>
Is this a problem with the vexpress (e.g. the timer runs at 24MHz
regardless of this setting, which would explain the otherwise odd
numbers you give above), or just that linux code assumes the timer is
always 24MHz?

If it''s the latter, I think it needs to be fixed on the linux side. 
Xen
can''t let the guest OS dictate things like this, since we might want to
run two guests with different OSes.  Also, if linux changes its choice,
we''d have trouble with old and new linux running together.

Tim.

On Mon, 24 Jun 2013, Tim Deegan wrote:
> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
> > On TC2, the timer frequency was set to 100 Mhz which slows down the
whole
> > platform.
> > When Linux is running on bare metal, the frequency is 24 Mhz.
> > 
> > "sleep 60" in dom0 takes:
> >   - 4 mins with a frequency equals to 100 Mhz
> >   - 1 min with a frequency equals to 24 Mhz
> > 
> > Signed-off-by: Julien Grall <julien.grall@linaro.org>
> 
> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
> regardless of this setting, which would explain the otherwise odd
> numbers you give above), or just that linux code assumes the timer is
> always 24MHz?
> 
> If it''s the latter, I think it needs to be fixed on the linux
side.  Xen
> can''t let the guest OS dictate things like this, since we might
want to
> run two guests with different OSes.  Also, if linux changes its choice,
> we''d have trouble with old and new linux running together.
Linux gets the frequency from:

1) device tree
2) the arch timer register

in this order.
Either way it should work.

Could it be that Xen is failing to set the frequency on the versatile
express? Therefore the hardware is running the timer at a different
frequency than Xen is expecting it to?

On 06/24/2013 03:52 PM, Stefano Stabellini wrote:
> On Mon, 24 Jun 2013, Tim Deegan wrote:
>> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
>>> On TC2, the timer frequency was set to 100 Mhz which slows down the
whole
>>> platform.
>>> When Linux is running on bare metal, the frequency is 24 Mhz.
>>>
>>> "sleep 60" in dom0 takes:
>>>   - 4 mins with a frequency equals to 100 Mhz
>>>   - 1 min with a frequency equals to 24 Mhz
>>>
>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>
>> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
>> regardless of this setting, which would explain the otherwise odd
>> numbers you give above), or just that linux code assumes the timer is
>> always 24MHz?
>>
>> If it''s the latter, I think it needs to be fixed on the linux
side.  Xen
>> can''t let the guest OS dictate things like this, since we
might want to
>> run two guests with different OSes.  Also, if linux changes its choice,
>> we''d have trouble with old and new linux running together.
> 
> Linux gets the frequency from:
> 
> 1) device tree
> 2) the arch timer register
> 
> in this order.
> Either way it should work.
> 
> Could it be that Xen is failing to set the frequency on the versatile
> express? Therefore the hardware is running the timer at a different
> frequency than Xen is expecting it to?


Lots of place in the DTS and the board configuration use a frequency of
24Mhz.
On the arndale, the timer frequency is 24Mhz, why do we need 100Mhz on
the versatile express?

It seems U-boot uses SP810 to modify the clock frequency on the
versatile express but I can''t find documentation about the different
registers.

-- 
Julien

On Mon, 24 Jun 2013, Julien Grall wrote:
> On 06/24/2013 03:52 PM, Stefano Stabellini wrote:
> 
> > On Mon, 24 Jun 2013, Tim Deegan wrote:
> >> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
> >>> On TC2, the timer frequency was set to 100 Mhz which slows
down the whole
> >>> platform.
> >>> When Linux is running on bare metal, the frequency is 24 Mhz.
> >>>
> >>> "sleep 60" in dom0 takes:
> >>>   - 4 mins with a frequency equals to 100 Mhz
> >>>   - 1 min with a frequency equals to 24 Mhz
> >>>
> >>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>
> >> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
> >> regardless of this setting, which would explain the otherwise odd
> >> numbers you give above), or just that linux code assumes the timer
is
> >> always 24MHz?
> >>
> >> If it''s the latter, I think it needs to be fixed on the
linux side.  Xen
> >> can''t let the guest OS dictate things like this, since we
might want to
> >> run two guests with different OSes.  Also, if linux changes its
choice,
> >> we''d have trouble with old and new linux running
together.
> > 
> > Linux gets the frequency from:
> > 
> > 1) device tree
> > 2) the arch timer register
> > 
> > in this order.
> > Either way it should work.
> > 
> > Could it be that Xen is failing to set the frequency on the versatile
> > express? Therefore the hardware is running the timer at a different
> > frequency than Xen is expecting it to?
> 
> 
> 
> Lots of place in the DTS and the board configuration use a frequency of
> 24Mhz.
> On the arndale, the timer frequency is 24Mhz, why do we need 100Mhz on
> the versatile express?
> 
> It seems U-boot uses SP810 to modify the clock frequency on the
> versatile express but I can''t find documentation about the
different
> registers.
In that case, if Xen is actually unable of changing the clock frequency,
just go ahead and remove:

        /* Ugly: the system timer''s frequency register is only
         * programmable in Secure state.  Since we don''t know where its
         * memory-mapped control registers live, we can''t find out the
         * right frequency. */
        mcr   CP32(r0, CNTFRQ)

On Mon, 2013-06-24 at 14:51 +0100, Tim Deegan wrote:
> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
> > On TC2, the timer frequency was set to 100 Mhz which slows down the
whole
> > platform.
> > When Linux is running on bare metal, the frequency is 24 Mhz.
> > 
> > "sleep 60" in dom0 takes:
> >   - 4 mins with a frequency equals to 100 Mhz
> >   - 1 min with a frequency equals to 24 Mhz
> > 
> > Signed-off-by: Julien Grall <julien.grall@linaro.org>
> 
> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
> regardless of this setting, which would explain the otherwise odd
> numbers you give above), or just that linux code assumes the timer is
> always 24MHz?
> 
> If it''s the latter, I think it needs to be fixed on the linux
side.  Xen
> can''t let the guest OS dictate things like this, since we might
want to
> run two guests with different OSes.  Also, if linux changes its choice,
> we''d have trouble with old and new linux running together.
Isn''t this timer only available to dom0, as the owner of most
peripherals?

Guests use the arch timers, because that is all they see.

Xen doesn''t use this timer at all for any purpose AFAIK.

That said, it is a bit of an odd quirk that dom0 can see platform timers
too, especially if it can cause breakage, but maybe not as critical as
all that?

Ian.

On 06/25/2013 04:31 PM, Ian Campbell wrote:
> On Mon, 2013-06-24 at 14:51 +0100, Tim Deegan wrote:
>> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
>>> On TC2, the timer frequency was set to 100 Mhz which slows down the
whole
>>> platform.
>>> When Linux is running on bare metal, the frequency is 24 Mhz.
>>>
>>> "sleep 60" in dom0 takes:
>>>   - 4 mins with a frequency equals to 100 Mhz
>>>   - 1 min with a frequency equals to 24 Mhz
>>>
>>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
>>
>> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
>> regardless of this setting, which would explain the otherwise odd
>> numbers you give above), or just that linux code assumes the timer is
>> always 24MHz?
>>
>> If it''s the latter, I think it needs to be fixed on the linux
side.  Xen
>> can''t let the guest OS dictate things like this, since we
might want to
>> run two guests with different OSes.  Also, if linux changes its choice,
>> we''d have trouble with old and new linux running together.
> 
> Isn''t this timer only available to dom0, as the owner of most
> peripherals?
> 
> Guests use the arch timers, because that is all they see.
> 
> Xen doesn''t use this timer at all for any purpose AFAIK.
> 
> That said, it is a bit of an odd quirk that dom0 can see platform timers
> too, especially if it can cause breakage, but maybe not as critical as
> all that?

Timers on ARM seems a bit complex. The current issue is with arch timer
(which is also used by dom0).

As I understand, there are also a bunch of clocks for different device
(UARTs, I2C,...). If theses clocks is not "pass-through" to dom0
nothing
will work.

-- 
Julien

On Tue, 2013-06-25 at 16:46 +0100, Julien Grall wrote:
> On 06/25/2013 04:31 PM, Ian Campbell wrote:
> 
> > On Mon, 2013-06-24 at 14:51 +0100, Tim Deegan wrote:
> >> At 14:10 +0100 on 24 Jun (1372083058), Julien Grall wrote:
> >>> On TC2, the timer frequency was set to 100 Mhz which slows
down the whole
> >>> platform.
> >>> When Linux is running on bare metal, the frequency is 24 Mhz.
> >>>
> >>> "sleep 60" in dom0 takes:
> >>>   - 4 mins with a frequency equals to 100 Mhz
> >>>   - 1 min with a frequency equals to 24 Mhz
> >>>
> >>> Signed-off-by: Julien Grall <julien.grall@linaro.org>
> >>
> >> Is this a problem with the vexpress (e.g. the timer runs at 24MHz
> >> regardless of this setting, which would explain the otherwise odd
> >> numbers you give above), or just that linux code assumes the timer
is
> >> always 24MHz?
> >>
> >> If it''s the latter, I think it needs to be fixed on the
linux side.  Xen
> >> can''t let the guest OS dictate things like this, since we
might want to
> >> run two guests with different OSes.  Also, if linux changes its
choice,
> >> we''d have trouble with old and new linux running
together.
> > 
> > Isn''t this timer only available to dom0, as the owner of most
> > peripherals?
> > 
> > Guests use the arch timers, because that is all they see.
> > 
> > Xen doesn''t use this timer at all for any purpose AFAIK.
> > 
> > That said, it is a bit of an odd quirk that dom0 can see platform
timers
> > too, especially if it can cause breakage, but maybe not as critical as
> > all that?
> 
> 
> Timers on ARM seems a bit complex. The current issue is with arch timer
> (which is also used by dom0).
Oh, sorry, I somehow thought we were talking about the platform timers,
never mind me!
> 
> As I understand, there are also a bunch of clocks for different device
> (UARTs, I2C,...). If theses clocks is not "pass-through" to dom0
nothing
> will work.
>

>
> Timers on ARM seems a bit complex. The current issue is with arch timer
> (which is also used by dom0).
Just out of curiosity, what is the difference between arch timer and
platform timer?

>
> As I understand, there are also a bunch of clocks for different device
> (UARTs, I2C,...). If theses clocks is not "pass-through" to dom0
nothing
> will work.
>
> --
> Julien
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

At 09:05 +0900 on 26 Jun (1372237555), Sengul Thomas
wrote:
> >
> > Timers on ARM seems a bit complex. The current issue is with arch
timer
> > (which is also used by dom0).
> 
> Just out of curiosity, what is the difference between arch timer and
> platform timer?
The arch timer has its interface specified in the CPU manual and
required (IIRC) for any processor that supports the virt extensions.
The platform timers are whatever other time sources come on the SoC
you''re running on.

Having thought about this a bit, ISTR that the register in Julien''s
patch doesn''t control anything -- it''s there to tell the arch
timer''s
users how fast that timer is going, so it ought to be set up by whatever
sets up the timer itself (turning on oscillators &c).

So the right thing to do is to move that write into more board-specific
start-of-day code (and eventually into the boot-wrapper), and rely on
the bootloader/firmware to have done it in the general case.  On a real
TC2 is that register already correct at (xen''s) boot time?

But presumably just changing it from 100KHz to 24KHz for all vexpresses
is wrong because that will break things on the software models in the
opposite way.

Tim.

On Wed, 26 Jun 2013, Tim Deegan wrote:
> At 09:05 +0900 on 26 Jun (1372237555), Sengul Thomas wrote:
> > >
> > > Timers on ARM seems a bit complex. The current issue is with arch
timer
> > > (which is also used by dom0).
> > 
> > Just out of curiosity, what is the difference between arch timer and
> > platform timer?
> 
> The arch timer has its interface specified in the CPU manual and
> required (IIRC) for any processor that supports the virt extensions.
> The platform timers are whatever other time sources come on the SoC
> you''re running on.
> 
> Having thought about this a bit, ISTR that the register in
Julien''s
> patch doesn''t control anything -- it''s there to tell the
arch timer''s
> users how fast that timer is going, so it ought to be set up by whatever
> sets up the timer itself (turning on oscillators &c).
> 
> So the right thing to do is to move that write into more board-specific
> start-of-day code (and eventually into the boot-wrapper), and rely on
> the bootloader/firmware to have done it in the general case.
I think that you are right

> On a real
> TC2 is that register already correct at (xen''s) boot time?
> 
> But presumably just changing it from 100KHz to 24KHz for all vexpresses
> is wrong because that will break things on the software models in the
> opposite way.
I think we should read CNTFRQ at boot time rather than trying to write
it. We should be able to cope with whatever frequency has been setup by
the firmware, right?

At 11:24 +0100 on 26 Jun (1372245863), Stefano Stabellini
wrote:
> On Wed, 26 Jun 2013, Tim Deegan wrote:
> > At 09:05 +0900 on 26 Jun (1372237555), Sengul Thomas wrote:
> > > >
> > > > Timers on ARM seems a bit complex. The current issue is with
arch timer
> > > > (which is also used by dom0).
> > > 
> > > Just out of curiosity, what is the difference between arch timer
and
> > > platform timer?
> > 
> > The arch timer has its interface specified in the CPU manual and
> > required (IIRC) for any processor that supports the virt extensions.
> > The platform timers are whatever other time sources come on the SoC
> > you''re running on.
> > 
> > Having thought about this a bit, ISTR that the register in
Julien''s
> > patch doesn''t control anything -- it''s there to tell
the arch timer''s
> > users how fast that timer is going, so it ought to be set up by
whatever
> > sets up the timer itself (turning on oscillators &c).
> > 
> > So the right thing to do is to move that write into more
board-specific
> > start-of-day code (and eventually into the boot-wrapper), and rely on
> > the bootloader/firmware to have done it in the general case.
> 
> I think that you are right
> 
> 
> > On a real
> > TC2 is that register already correct at (xen''s) boot time?
> > 
> > But presumably just changing it from 100KHz to 24KHz for all
vexpresses
> > is wrong because that will break things on the software models in the
> > opposite way.
> 
> I think we should read CNTFRQ at boot time rather than trying to write
> it. We should be able to cope with whatever frequency has been setup by
> the firmware, right?
Yes -- in the case where the firmware has done it, we should be reading
this register rather than writing it. 

On the software VE model the code was developed on, there is no firmware
and so Xen has to set this up along with the other things that we''d
expect firmware to do.  So we can move that write to the start-fo-day
board-specific asm hacks, and the actual driver code can just read it.

Cheers,

Tim.

On 06/26/2013 11:52 AM, Tim Deegan wrote:
> At 11:24 +0100 on 26 Jun (1372245863), Stefano Stabellini wrote:
>> On Wed, 26 Jun 2013, Tim Deegan wrote:
>>> At 09:05 +0900 on 26 Jun (1372237555), Sengul Thomas wrote:
>>>>>
>>>>> Timers on ARM seems a bit complex. The current issue is
with arch timer
>>>>> (which is also used by dom0).
>>>>
>>>> Just out of curiosity, what is the difference between arch
timer and
>>>> platform timer?
>>>
>>> The arch timer has its interface specified in the CPU manual and
>>> required (IIRC) for any processor that supports the virt
extensions.
>>> The platform timers are whatever other time sources come on the SoC
>>> you''re running on.
>>>
>>> Having thought about this a bit, ISTR that the register in
Julien''s
>>> patch doesn''t control anything -- it''s there to
tell the arch timer''s
>>> users how fast that timer is going, so it ought to be set up by
whatever
>>> sets up the timer itself (turning on oscillators &c).
>>>
>>> So the right thing to do is to move that write into more
board-specific
>>> start-of-day code (and eventually into the boot-wrapper), and rely
on
>>> the bootloader/firmware to have done it in the general case.
>>
>> I think that you are right
>>
>>
>>> On a real
>>> TC2 is that register already correct at (xen''s) boot time?
>>>
>>> But presumably just changing it from 100KHz to 24KHz for all
vexpresses
>>> is wrong because that will break things on the software models in
the
>>> opposite way.
>>
>> I think we should read CNTFRQ at boot time rather than trying to write
>> it. We should be able to cope with whatever frequency has been setup by
>> the firmware, right?
> 
> Yes -- in the case where the firmware has done it, we should be reading
> this register rather than writing it. 
Xen already reads the timer frequency in arm/time.c.
> On the software VE model the code was developed on, there is no firmware
> and so Xen has to set this up along with the other things that
we''d
> expect firmware to do.  So we can move that write to the start-fo-day
> board-specific asm hacks, and the actual driver code can just read it.

Is it possible to start to use u-boot on the VE model? So we can remove
the code which set up the timer freqency.

-- 
Julien

On Wed, 2013-06-26 at 12:10 +0100, Julien Grall wrote:
> Is it possible to start to use u-boot on the VE model?
If it is possible to do on the real VE then, in principal, probably. But
I get the impression that it is odd to do so and a bit faffsome even on
the real VE.
> So we can remove the code which set up the timer freqency.
I would love this!

Ian.
>

On 06/26/2013 12:39 PM, Ian Campbell wrote:
> On Wed, 2013-06-26 at 12:10 +0100, Julien Grall wrote:
> 
>> Is it possible to start to use u-boot on the VE model?
> 
> If it is possible to do on the real VE then, in principal, probably. But
> I get the impression that it is odd to do so and a bit faffsome even on
> the real VE.
Ubuntu ARM already uses u-boot on the versatile express. If U-boot is
configure to retrieve Xen and Linux via tftp I don''t see specific issue
for the development.

I will give a try when I will have time.

-- 
Julien

At 13:38 +0100 on 26 Jun (1372253902), Julien Grall
wrote:
> On 06/26/2013 12:39 PM, Ian Campbell wrote:
> 
> > On Wed, 2013-06-26 at 12:10 +0100, Julien Grall wrote:
> > 
> >> Is it possible to start to use u-boot on the VE model?
> > 
> > If it is possible to do on the real VE then, in principal, probably.
But
> > I get the impression that it is odd to do so and a bit faffsome even
on
> > the real VE.
> 
> Ubuntu ARM already uses u-boot on the versatile express. If U-boot is
> configure to retrieve Xen and Linux via tftp I don''t see specific
issue
> for the development.
Sounds great!  It should let us get rid of a lot of other start-of-day
hacks too.

Tim.