(sorry for not in-lining this...)
mmm.... major swapping...
Note that disk performance is an order of magnitude (or worse! ;) slower
than memory. Memory is nano-second response and disk is tens of
milliseconds on average or worse.
So - I''m not the least bit surprised!
Unfortunately, this is clearly getting a little O/T for dtrace-discuss
though... Sorry about that, DTracers...
Anyhoo - OpenSolaris makes some assumptions around the capabilities of
the system expected to be running. Namely that it''ll have what would be
the memory capacity of a modern system.... ;) Getting rid of all the X
and Gnome stuff will help, but nothing will help you (short term) in
terms of reducing the memory usage of the pkg stuff...
For what it''s worth, my worst ''play'' box these days
has 2GB of memory!
Maybe you could hop on over to pkg-discuss and ask the good folks there
if there is anything special you could do for a memory-challenged system.
Cheers!
Nathan.
On 10/03/09 10:57 AM, Maarten Dirkse wrote:> Thanks! This is really helpful stuff.
>
> This is the output of several iterations of ?vmstat 1?:
>
> kthr memory page disk faults cpu
>
> r b w swap free re mf pi po fr de sr cd cd cd s2 in sy cs us
> sy id
>
> 0 0 56 981160 56772 4 76 236 259 270 0 594 79 0 0 -0 374 144 11347 2
> 24 75
>
> 0 0 0 0 0 6 99 240 808 816 0 1359 132 0 0 0 437 154 13733 1
> 33 66
>
> 0 0 51 841488 4664 0 82 328 852 856 0 1326 137 0 0 0 495 86 17229 0
> 35 65
>
> 0 0 51 841488 4080 0 121 484 168 168 0 370 148 0 0 0 547 84 23482 1
> 49 50
>
> 0 0 51 841424 4192 4 101 388 328 328 0 405 133 0 0 0 484 79 19058 0
> 37 63
>
> 1 0 51 841448 4188 11 97 344 480 480 0 476 144 0 0 0 533 89 17460 1
> 38 61
>
> 0 0 51 841472 4036 2 95 372 488 488 0 406 101 0 0 0 425 83 18838 1
> 36 63
>
> 0 0 51 841476 4008 8 115 428 448 452 0 616 113 0 0 0 450 81 21406 1
> 40 59
>
> 0 0 51 841476 3948 2 103 404 222 222 0 725 137 0 0 0 510 79 19796 1
> 44 55
>
> 0 0 51 841592 3912 4 115 444 626 626 0 1093 142 0 0 0 516 87 21491 0
> 41 59
>
> 1 0 51 841600 3992 7 153 586 250 250 0 508 152 0 0 0 522 82 27576 1
> 50 49
>
> 0 0 51 841664 4160 3 112 436 347 352 0 416 166 0 0 0 580 83 21353 0
> 45 55
>
> 0 0 51 841680 4040 2 132 521 339 352 0 363 140 0 0 0 508 83 25530 0
> 46 54
>
> 0 0 51 841684 3988 3 143 558 543 558 0 558 143 0 0 0 507 81 26296 1
> 50 50
>
> 0 0 51 841692 4128 3 83 292 340 340 0 440 138 0 0 0 524 82 15242 0
> 36 64
>
> 0 0 51 842240 4008 5 95 277 0 0 0 0 89 0 0 0 426 147 15591 1
> 32 67
>
> 0 0 51 842392 3916 4 90 319 1075 1127 0 3046 114 0 0 0 484 310 17869
> 0 42 58
>
> 0 0 51 841340 4284 5 37 128 980 1016 0 1419 114 0 0 0 470 193 7636 1
> 24 75
>
> It seems as though memory shortage is the issue, although I?m surprised
> that it slows things down dat much, and that pkg takes up so much memory
> (see below). I?ve actually already killed a whole bunch of services
> (anything to do with X, and a bunch of other stuff), and I?m looking at
> Milax to seen what else I can safely kill. I don?t suppose you know of
> any way to assign all memory to pkg, or to make pkg more economical in
> it?s memory usage?
>
> Regards,
>
> Maarten
>
> By the way, this is the top output of prstat -s rss and prstat -s size
> respectively.
>
> PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
>
> 1268 root 111M 64M run 33 0 0:01:14 9.3% pkg/1
>
> 1313 maarten 6080K 3016K cpu0 59 0 0:00:00 0.2% prstat/1
>
> 1265 maarten 82M 2564K sleep 59 0 0:02:36 0.2% java/14
>
> 115 root 8692K 1764K sleep 59 0 0:00:17 0.1% nscd/30
>
> 1293 maarten 5896K 1520K sleep 59 0 0:00:00 0.0% bash/1
>
> 1290 maarten 10M 1496K sleep 59 0 0:00:00 0.0% sshd/1
>
> 474 root 18M 1252K sleep 59 0 0:00:03 0.0% fmd/17
>
> 1251 maarten 10M 1192K sleep 59 0 0:00:22 0.0% sshd/1
>
> 49 root 12M 1172K sleep 59 0 0:00:05 0.0% devfsadm/6
>
>
> PID USERNAME SIZE RSS STATE PRI NICE TIME CPU PROCESS/NLWP
>
> 1268 root 130M 64M run 58 0 0:01:21 2.4% pkg/1
>
> 1265 maarten 82M 1612K sleep 59 0 0:02:36 0.2% java/14
>
> 474 root 18M 1228K sleep 59 0 0:00:03 0.0% fmd/17
>
> 7 root 13M 552K sleep 59 0 0:00:09 0.0% svc.startd/12
>
> 9 root 12M 732K sleep 59 0 0:00:47 0.0% svc.configd/25
>
> 49 root 12M 1172K sleep 59 0 0:00:05 0.0% devfsadm/6
>
> 1251 maarten 10M 1092K sleep 59 0 0:00:22 0.0% sshd/1
>
> 1006 maarten 10M 268K sleep 59 0 0:00:02 0.0% sshd/1
>
> 1290 maarten 10M 1596K sleep 59 0 0:00:00 0.1% sshd/1
>
> 116 daemon 9404K 1136K sleep 59 0 0:00:03 0.0% kcfd/5
>
>
>
> -----Original Message-----
> From: Nathan.Kroenert at Sun.COM [mailto:Nathan.Kroenert at Sun.COM]
> Sent: dinsdag 10 maart 2009 0:47
> To: Maarten Dirkse
> Cc: dtrace-discuss at opensolaris.org
> Subject: Re: [dtrace-discuss] pkg commands very slow
>
> best to let it run for a few seconds to get some real data.
>
> The first line of data is just averages since boot time.
>
> vmstat 2 will keep running till you hit ctrl-c, printing one line every
>
> 2 seconds.
>
> Scanning at all indicates that we are running low on memory - and would
>
> thus be paging, which will slow things to a crawl.
>
> Also note that the ''w'' column (third column) is the
number of threads
>
> swapped out - indicating a dire shortage of memory.
>
> So - What you *could* do if you wanted to persist with only 256 MB of
>
> memory is to look at what is actually using memory that you don''t
need.
>
> I''d usually use prstat -s rss to see who is *actually* using
memory and
>
> prstat -s size to see who is using lots of virtual memory.
>
> From there, you could disable services you don''t need to be
running -
>
> for which I''d be using svcadm disable <service name>
>
> All this being said, I don''t really know how much effort has been
made
>
> to get opensolaris running in a small memory footprint (particularly
>
> with a GUI running, like Gnome.)
>
> Perhaps changing window managers might help... ;)
>
> Cheers!
>
> Nathan.
>
>
>
> On 10/03/09 10:38 AM, Maarten Dirkse wrote:
>
>> This is the output of vmstat when pkg is stuck at "Creating
plan":
>
>>
>
>> kthr memory page disk
>
>> faults cpu
>
>> r b w swap free re mf pi po fr de sr cd cd cd s2
in
>
>> sy cs us sy id
>
>> 0 0 56 982408 57360 4 76 234 257 268 0 589 79 0 0 -0
> 372 144
>
>> 11258 2 23 75
>
>>
>
>> There seems to be 57mb of memory free, and the scanrate is 589.
>
>>
>
>> -----Original Message-----
>
>> From: Nathan.Kroenert at Sun.COM [mailto:Nathan.Kroenert at Sun.COM]
>
>> Sent: dinsdag 10 maart 2009 0:26
>
>> To: Maarten
>
>> Cc: dtrace-discuss at opensolaris.org
>
>> Subject: Re: [dtrace-discuss] pkg commands very slow
>
>>
>
>> I can''t help but feel that you will be choking on a dire lack
of memory.
>
>>
>
>> Next time you are running a pkg command, take a poke at a vmstat and
>
>> look at the free memory column and the scanrate... if freemem is close
>
>> to 0, and scanrate is not, you have likely found your bunny.
>
>>
>
>> Cheers!
>
>>
>
>> Nathan.
>
>>
>
>> On 10/03/09 09:20 AM, Maarten wrote:
>
>> > Hi,
>
>> >
>
>> > Because I wanted to use ZFS, I installed OpenSolaris (2008.11)on
an
> old HP
>
>> Vectra Pentium III with 256mb of RAM. Because I couldn''t
install using the
>
>> machine itself (not enough memory), I transplanted the hard drive to
> another
>
>> machine with more memory, installed Osol, and then put the hd back in
the
>
>> old box. This worked like a charm and I''m very pleased,
except for one
>
>> thing: every pkg command is excruciatingly slow. The first use I made
> of it
>
>> (pkg install IPSnano from the blastwave repository) resulted in it
> building
>
>> its index, which took 4 hours(!). Subsequent use of "pkg
install" was
>
>> faster, but "pkg install IPSnmap" still took something like
20-30 minutes.
>
>> > Most of the time it appears stuck at the "Creating plan"
phase, during
>
>> which there is constant disk activity. I''m pretty sure
it''s not just
> down to
>
>> the machine being slow (although it''s definitely no speed
demon) as
> all the
>
>> other software seems to run at reasonable speeds. The network
isn''t slow
>
>> (wget works at expected speeds), and the disk perform as expected when
> using
>
>> different apps than pkg.
>
>> > Does anyone have any ideas as to why this would be so slow? Or
maybe a
>
>> handy dtrace script that I can use to diagnose the problem
(I''m a bit of a
>
>> dtrace n00b).
>
>> > Thanks in advance,
>
>> > Maarten
>
>>
>
> --
>
>
> //////////////////////////////////////////////////////////////////
>
> // Nathan Kroenert nathan.kroenert at sun.com //
>
> // Senior Systems Engineer Phone: +61 3 9869 6255 //
>
> // Global Systems Engineering Fax: +61 3 9869 6288 //
>
> // Level 7, 476 St. Kilda Road //
>
> // Melbourne 3004 Victoria Australia //
>
> //////////////////////////////////////////////////////////////////
>
--
//////////////////////////////////////////////////////////////////
// Nathan Kroenert nathan.kroenert at sun.com //
// Senior Systems Engineer Phone: +61 3 9869 6255 //
// Global Systems Engineering Fax: +61 3 9869 6288 //
// Level 7, 476 St. Kilda Road //
// Melbourne 3004 Victoria Australia //
//////////////////////////////////////////////////////////////////