crossbow discuss - Dec 2007 - Questions on Solaris zone CPU shceduler sensitivity

Hi guys,

We are working on using Crossbow for a project, where we are trying to 
host game servers in a zone. Our objective is to show that one can host 
game servers in VMs similar to other typical applications like web 
server and get benefits of consolidation, QoS control etc.

However, to achieve that, one of the primary requirements is that the 
response time from the game server should be at least as good as running 
the game server on a normal machine. Some of the FPS games require that 
a client should never experience more than 100millisec response time 
(while waiting for a game update etc.)

We have been experimenting with Zones to figure out whether we can 
achieve this or not. From the documentation, we see that current Solaris 
CPU scheduler schedules 100ms slices to zones, and it is quite possible 
for a zone to not get CPU share for a few hundreds of milli seconds. We 
also  know that we can reduce the granularity of scheduling to 20ms.

Our questions are:

1. Is it still possible to meet the 100ms deadline? It is still possible 
that a VM doesn''t get scheduled with in 100ms?
2. If we can not guarantee it, would it be better, if we develop a 
scheduler like EDF-scheduler, that''s done, in Xen for example.
3. Are there other schedulers available other than fair-share scheduler?

Apart from that, we also would like to know, if the following is 
possible using Crossbow.

Is it possible to classify packets in a single flow differently? I 
understand that flowadm allows setting up priorities, bandwidth shares 
etc. for a flow, but can it done to DIFFERENT packets in a single flow? 
For example, a game server may prioritize between ''action''
packets (say
shoot, kill etc.) and ''terrain'' packets (say a new landscape)
from a
single client.

Any comments are appreciated.

Thanks,
Pradeep

Hi Pradeep,

Pradeep Padala wrote:
> Hi guys,
> 
> We are working on using Crossbow for a project, where we are trying to 
> host game servers in a zone. Our objective is to show that one can host 
> game servers in VMs similar to other typical applications like web 
> server and get benefits of consolidation, QoS control etc.
> 
> However, to achieve that, one of the primary requirements is that the 
> response time from the game server should be at least as good as running 
> the game server on a normal machine. Some of the FPS games require that 
> a client should never experience more than 100millisec response time 
> (while waiting for a game update etc.)
> 
> We have been experimenting with Zones to figure out whether we can 
> achieve this or not. From the documentation, we see that current Solaris 
> CPU scheduler schedules 100ms slices to zones, and it is quite possible 
> for a zone to not get CPU share for a few hundreds of milli seconds. We 
> also  know that we can reduce the granularity of scheduling to 20ms.
> 
> Our questions are:
> 
> 1. Is it still possible to meet the 100ms deadline? It is still possible 
> that a VM doesn''t get scheduled with in 100ms?
Zones are not like VMs. Zones is an abstraction of [a subset of] 
processes, and all the processes on the system are handled by a single 
scheduler. Scheduling decisions are made based on priority, shares (if 
using FSS), resource pools/psets, and the like. Whether a process gets 
CPU depends not on scheduling the "VM" or zone, but how it relates to 
other processes that it needs to contend with for CPU. This set of other 
processes can be determined by using some of the resoruce management 
tools in Solaris, including pools, FSS, other scheduling classes, 
interrupt distribution.
> 2. If we can not guarantee it, would it be better, if we develop a 
> scheduler like EDF-scheduler, that''s done, in Xen for example.
Xen and other virtualization techniques like it may have to resort to this.
> 3. Are there other schedulers available other than fair-share scheduler?
Fixed and realtime are two. But don''t use RT unless you multiple CPUs. 
Also consider pools if you have sufficient CPUs/cores/strands (if using 
CMT).

I don''t believe you can put a process into the realtime class from with
a non-global zone. Since it have very high priority, that could be used 
to affect the whole system. You would have to do so from the global 
zone. however, I don''t think realtime would be a good idea here--you 
probably have too many processes and could have a dead-lock situation.

rm-discuss may be a good place to also ask.
> Apart from that, we also would like to know, if the following is 
> possible using Crossbow.
> 
> Is it possible to classify packets in a single flow differently? I 
> understand that flowadm allows setting up priorities, bandwidth shares 
> etc. for a flow, but can it done to DIFFERENT packets in a single flow? 
> For example, a game server may prioritize between
''action'' packets (say
> shoot, kill etc.) and ''terrain'' packets (say a new
landscape) from a
> single client.
Interesting question, looking higher up into the packet. Don''t know the
answer to that.

Steffen
> 
> Any comments are appreciated.
> 
> Thanks,
> Pradeep
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss

Thanks for the clarifications. A few comments below.
> > 1. Is it still possible to meet the 100ms deadline? It is still 
> > possible that a VM doesn''t get scheduled with in 100ms?
> 
> Zones are not like VMs. Zones is an abstraction of [a subset 
> of] processes, and all the processes on the system are 
> handled by a single scheduler. Scheduling decisions are made 
> based on priority, shares (if using FSS), resource 
> pools/psets, and the like. Whether a process gets CPU depends 
> not on scheduling the "VM" or zone, but how it relates to 
> other processes that it needs to contend with for CPU. This 
> set of other processes can be determined by using some of the 
> resoruce management tools in Solaris, including pools, FSS, 
> other scheduling classes, interrupt distribution.
Say, we have two zones running two network intensive applications. Each zone
is pinned to a single CPU. Since, both want the CPU (for processing the
network packets) and want it for the full slice, scheduling matters. How do
I figure out the worst case scenario for NOT getting scheduled for a period
of time? What are the options other than the granularity, that I can use to
improve the sensitivity.

We are using FSS right now, and I believe FSS is used to schedule the zones.
The ''fixed'' and ''realtime'' schedulers you
mentioned are for processes,
right? For our purposes, we will mostly have one single process in a zone,
so better scheduling for zones is what we need.
 
Pradeep

Pradeep Padala wrote:
> Thanks for the clarifications. A few comments below.
> 
>>> 1. Is it still possible to meet the 100ms deadline? It is still 
>>> possible that a VM doesn''t get scheduled with in 100ms?
>> Zones are not like VMs. Zones is an abstraction of [a subset 
>> of] processes, and all the processes on the system are 
>> handled by a single scheduler. Scheduling decisions are made 
>> based on priority, shares (if using FSS), resource 
>> pools/psets, and the like. Whether a process gets CPU depends 
>> not on scheduling the "VM" or zone, but how it relates to 
>> other processes that it needs to contend with for CPU. This 
>> set of other processes can be determined by using some of the 
>> resoruce management tools in Solaris, including pools, FSS, 
>> other scheduling classes, interrupt distribution.
> 
> Say, we have two zones running two network intensive applications. Each
zone
> is pinned to a single CPU. Since, both want the CPU (for processing the
Are you pinning the CPU or are you wishing to provide exclusive access 
to a CPU per process?
> network packets) and want it for the full slice, scheduling matters. How do
I am not a scheduler expert, but when a process does I/O, the scheduler 
will re-access things.
> I figure out the worst case scenario for NOT getting scheduled for a period
> of time? What are the options other than the granularity, that I can use to
> improve the sensitivity.
> 
> We are using FSS right now, and I believe FSS is used to schedule the
zones.
Not exactly. FSS is used to schedule all the processes in the zones 
based on the shares attributed to the zone when it was started. The zone 
does not run--the processes in a zone run.
> The ''fixed'' and ''realtime'' schedulers
you mentioned are for processes,
> right? For our purposes, we will mostly have one single process in a zone,
> so better scheduling for zones is what we need.
What build are you running? And how many CPUs?

I don''t know whether you can assign FSS to a zone for general
processing
and make a subset of processes run in a different class, such as FX or 
RT. And whether you can put the process into FX from within the zone 
(should not be able to put it into RT).

Steffen
>  
> Pradeep
> 
> _______________________________________________
> crossbow-discuss mailing list
> crossbow-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/crossbow-discuss