thr3ads.net - Xen devel - xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0 [Nov 2013]

If this information is useful, please help other people find it:
Share via:

Sander Eikelenboom

2013-Nov-19 13:02 UTC

xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Hi Wei,

I ran into the following problem when trying to boot another guest after less
than a day of uptime.
(the system started 15 guests at boot already which went fine). dom0 is
allocated a fixed 1536M.

Both host as pv guests run the same kernel, some hvm''s run a slightly
older kernel (3.9 f.e.)

The are quite some granttable messages in xl dmesg, i also included these and a
"vmstat -m"

--
Sander

[54807.299791] xenwatch: page allocation failure: order:4, mode:0x10c0d0
[54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted 3.12.0-20131104 #1
[54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS V1.8B1
09/13/2010
[54807.351990]  0000000000000000 ffff8800591a3828 ffffffff81a6a8ba
0000000000000006
[54807.368857]  000000000010c0d0 ffff8800591a38b8 ffffffff81138770
0000000000000004
[54807.385539]  000000000010c0d0 ffff8800591a38b8 ffffffff81a661de
ffff88005fd1c680
[54807.401978] Call Trace:
[54807.418015]  [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
[54807.433971]  [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
[54807.449644]  [<ffffffff81a661de>] ?
__alloc_pages_direct_compact+0x1ac/0x1be
[54807.465164]  [<ffffffff8113bfaa>] __alloc_pages_nodemask+0x7aa/0x9d0
[54807.480510]  [<ffffffff810ed069>] ?
trace_hardirqs_off_caller+0xb9/0x160
[54807.495622]  [<ffffffff81175277>] alloc_pages_current+0xb7/0x180
[54807.510530]  [<ffffffff81138059>] __get_free_pages+0x9/0x40
[54807.525185]  [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
[54807.539538]  [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
[54807.553814]  [<ffffffff8192ac20>] ? alloc_etherdev_mqs+0x20/0x20
[54807.567777]  [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
[54807.581473]  [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
[54807.594971]  [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
[54807.608231]  [<ffffffff81615105>] driver_probe_device+0x75/0x210
[54807.621227]  [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
[54807.634071]  [<ffffffff8161539b>] __device_attach+0x4b/0x60
[54807.646626]  [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
[54807.658918]  [<ffffffff81615058>] device_attach+0x98/0xb0
[54807.671253]  [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
[54807.683379]  [<ffffffff81612277>] device_add+0x3b7/0x700
[54807.695145]  [<ffffffff8161ca1d>] ? device_pm_sleep_init+0x4d/0x80
[54807.706824]  [<ffffffff816125d9>] device_register+0x19/0x20
[54807.718145]  [<ffffffff814bf9b1>] xenbus_probe_node+0x141/0x170
[54807.729256]  [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
[54807.740091]  [<ffffffff814bfbb0>] xenbus_dev_changed+0x1d0/0x1e0
[54807.750811]  [<ffffffff814bff26>] backend_changed+0x16/0x20
[54807.761256]  [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
[54807.771371]  [<ffffffff810bc1e0>] ? __init_waitqueue_head+0x60/0x60
[54807.781443]  [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
[54807.791310]  [<ffffffff810bb716>] kthread+0xd6/0xe0
[54807.800794]  [<ffffffff81a752bb>] ? _raw_spin_unlock_irq+0x2b/0x70
[54807.810150]  [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
[54807.819526]  [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
[54807.828638]  [<ffffffff810bb640>] ? __init_kthread_worker+0x70/0x70
[54807.837482] Mem-Info:
[54807.846011] Node 0 DMA per-cpu:
[54807.854277] CPU    0: hi:    0, btch:   1 usd:   0
[54807.862404] CPU    1: hi:    0, btch:   1 usd:   0
[54807.870241] CPU    2: hi:    0, btch:   1 usd:   0
[54807.877985] CPU    3: hi:    0, btch:   1 usd:   0
[54807.885344] CPU    4: hi:    0, btch:   1 usd:   0
[54807.892389] CPU    5: hi:    0, btch:   1 usd:   0
[54807.899102] Node 0 DMA32 per-cpu:
[54807.905665] CPU    0: hi:  186, btch:  31 usd:  68
[54807.911985] CPU    1: hi:  186, btch:  31 usd:   6
[54807.917964] CPU    2: hi:  186, btch:  31 usd: 149
[54807.923666] CPU    3: hi:  186, btch:  31 usd:  82
[54807.929031] CPU    4: hi:  186, btch:  31 usd: 169
[54807.934293] CPU    5: hi:  186, btch:  31 usd:   0
[54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
[54807.939207]  active_file:70321 inactive_file:177850 isolated_file:0
[54807.939207]  unevictable:562 dirty:34 writeback:0 unstable:0
[54807.939207]  free:31143 slab_reclaimable:21805 slab_unreclaimable:12717
[54807.939207]  mapped:3344 shmem:276 pagetables:1211 bounce:0
[54807.939207]  free_cma:0
[54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB
active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB
unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB
managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB
slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB
pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
[54807.981676] lowmem_reserve[]: 0 1395 1395 1395
[54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB high:7120kB
active_anon:51028kB inactive_anon:35772kB active_file:281280kB
inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB
present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB
mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB
kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[54808.011107] lowmem_reserve[]: 0 0 0 0
[54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM) 14*32kB (UEM)
8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (R)
0*4096kB = 5644kB
[54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB (EMR)
6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB =
118608kB
[54808.044303] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0
hugepages_size=2048kB
[54808.051507] 249203 total pagecache pages
[54808.058647] 201 pages in swap cache
[54808.065986] Swap cache stats: add 2902, delete 2701, find 62055/62191
[54808.073216] Free swap  = 2092752kB
[54808.080397] Total swap = 2097148kB
[54808.093896] 393215 pages RAM
[54808.101004] 30940 pages reserved
[54808.108168] 339108 pages shared
[54808.115202] 258777 pages non-shared
[54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0
[54808.129391] vif vif-16-0: 12 creating interface


~# vmstat -m
Cache                       Num  Total   Size  Pages
ext4_groupinfo_4k         24426  24426    216     18
ceph_osd_request              0      0    960     17
xt_hashlimit                  0      0    136     30
nf_conntrack_ffffffff822d2900   1950   2100    400     20
nf_conntrack_expect           0      0    288     28
dm_snap_pending_exception   2496   2496    104     39
kcopyd_job                   91    133   4176      7
dm_rq_target_io               0      0    416     19
search                        0      0    616     26
blkif_cache                 108    108    864     18
cfq_io_cq                   204    204    120     34
cfq_queue                   170    170    232     17
bsg_cmd                       0      0    312     26
ceph_cap                      0      0    128     32
ceph_inode_info               0      0   2176     15
gfs2_mblk                     0      0    432     18
gfs2_bufdata                  0      0     88     46
gfs2_inode                    0      0   1280     25
gfs2_glock(aspace)            0      0    960     17
gfs2_glock                    0      0    592     27
btrfs_delayed_data_ref        0      0     96     42
btrfs_delayed_ref_head        0      0    232     17
btrfs_delayed_node            0      0    392     20
btrfs_ordered_extent          0      0    472     17
btrfs_extent_buffer           0      0    568     28
btrfs_delalloc_work           0      0    184     22
btrfs_path                    0      0    144     28
btrfs_transaction             0      0    600     27
btrfs_trans_handle            0      0    160     25
btrfs_inode                   0      0   1944     16
fuse_request                102    102    480     17
fuse_inode                  336    476   1152     28
ntfs_big_inode_cache          0      0   1664     19
ntfs_inode_cache              0      0    696     23
cifs_small_rq                36     36    448     18
cifs_request                  4      4  16512      1
cifs_inode_cache              0      0   1112     29
isofs_inode_cache             0      0    960     17
fat_inode_cache               0      0   1168     28
fat_cache                     0      0     40    102
hugetlbfs_inode_cache        16     16    976     16
jbd2_transaction_s          150    150    320     25
jbd2_journal_handle         306    306     80     51
journal_handle                0      0     56     73
journal_head               1518   1800    112     36
revoke_table               1536   1536     16    256
revoke_record               768    768     32    128
ext4_inode_cache          30702  30952   1704     19
ext4_free_data              768    768     64     64
ext4_allocation_context     180    180    136     30
ext4_prealloc_space         156    156    152     26
ext4_io_end                 336    336     72     56
ext4_extent_status         7361   7446     40    102
ext3_inode_cache              0      0   1312     24
ext3_xattr                 3726   3956     88     46
dquot                         0      0    384     21
kioctx                        0      0    896     18
pid_namespace                 0      0   2208     14
posix_timers_cache          108    108    296     27
UNIX                        198    198   1472     22
Cache                       Num  Total   Size  Pages
UDP-Lite                      0      0   1280     25
ip_fib_trie                 438    438     56     73
PING                          0      0   1216     26
UDP                         150    150   1280     25
tw_sock_TCP                 399    399    192     21
TCP                         104    104   2368     13
fscache_cookie_jar            0      0    192     21
sgpool-128                   54     78   5120      6
sgpool-64                    72     72   2560     12
sgpool-32                   325    325   1280     25
sgpool-16                   277    350    640     25
blkdev_integrity              0      0    112     36
blkdev_queue                165    165   2872     11
blkdev_requests             315    315    376     21
fsnotify_event_holder         0      0     24    170
sock_inode_cache            272    272    960     17
file_lock_cache             102    102    240     17
shmem_inode_cache          2019   2072   1144     28
Acpi-State                  255    255     80     51
Acpi-Namespace             1428   1428     40    102
task_delay_info            1704   1704    168     24
taskstats                   144    144    328     24
proc_inode_cache           1632   1664    976     16
sigqueue                    250    250    160     25
bdev_cache                  192    192   1344     24
sysfs_dir_cache           23817  23968    144     28
filp                       1807   2400    320     25
inode_cache                1861   2244    912     17
dentry                    27252  29440    248     16
buffer_head               67283 102063    104     39
vm_area_struct             4527   4708    184     22
mm_struct                   308    308   1152     28
files_cache                 253    253    704     23
signal_cache                667    667   1408     23
sighand_cache               407    490   2240     14
task_struct                 430    442   4240      7
anon_vma                   4676   4676    144     28
shared_policy_node          850    850     48     85
numa_policy                3528   3528     72     56
radix_tree_node            7622   8050    568     28
idr_layer_cache             240    240   2112     15
dma-kmalloc-8192              0      0   8192      4
dma-kmalloc-4096              0      0   4096      8
dma-kmalloc-2048              0      0   2048     16
dma-kmalloc-1024              0      0   1024     16
dma-kmalloc-512               0      0    512     16
dma-kmalloc-256               0      0    256     16
dma-kmalloc-128               0      0    128     32
dma-kmalloc-64                0      0     64     64
dma-kmalloc-32                0      0     32    128
dma-kmalloc-16                0      0     16    256
dma-kmalloc-8                 0      0      8    512
dma-kmalloc-192               0      0    192     21
dma-kmalloc-96                0      0     96     42
kmalloc-8192                 32     32   8192      4
kmalloc-4096                835    916   4096      8
kmalloc-2048                773    832   2048     16
kmalloc-1024               2335   2336   1024     16
kmalloc-512                1488   1536    512     16
kmalloc-256                2535   2784    256     16
Cache                       Num  Total   Size  Pages
kmalloc-192               69883  70707    192     21
kmalloc-128                2603   4992    128     32
kmalloc-96                 5628   5628     96     42
kmalloc-64                41984  41984     64     64
kmalloc-32               154792 156288     32    128
kmalloc-16                 6343   7936     16    256
kmalloc-8                  8192   8192      8    512
kmem_cache_node             256    256    128     32
kmem_cache                  176    176    256     16


(XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1) grant table
from (4) to (5) frames.
(XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack size to 8
frames
(XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant table
from (7) to (8) frames.
(XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 9
frames
(XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to 10
frames
(XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1) grant table
from (8) to (9) frames.
(XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1) grant table
from (9) to (10) frames.
(XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1) grant table
from (10) to (11) frames.
(XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack size to 11
frames
(XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack size to 12
frames
(XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12) grant
table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack size to 13
frames
(XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant
table from (4) to (5) frames.
(XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant
table from (5) to (6) frames.
(XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack size to 14
frames
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 15
frames
(XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to 16
frames
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 17
frames
(XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to 18
frames
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack size to 19
frames
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack size to 20
frames
(XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 21
frames
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to 22
frames
(XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 23
frames
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 24
frames
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 25
frames
(XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to 26
frames
(XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10) grant
table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant
table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant
table from (6) to (7) frames.
(XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack size to 27
frames
(XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack size to 28
frames
(XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant
table from (4) to (5) frames.
(XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant
table from (5) to (6) frames.
(XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack size to 29
frames
(XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack size to 30
frames
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table
from (4) to (5) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table
from (5) to (6) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant table
from (6) to (7) frames.
(XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack size to 31
frames
(XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m table =
0x52d3c5

Wei Liu

2013-Nov-19 13:23 UTC

head link

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

On Tue, Nov 19, 2013 at 02:02:57PM +0100, Sander Eikelenboom
wrote:> Hi Wei,
> 
> I ran into the following problem when trying to boot another guest after
less than a day of uptime.
> (the system started 15 guests at boot already which went fine). dom0 is
allocated a fixed 1536M.
> 
> Both host as pv guests run the same kernel, some hvm''s run a
slightly older kernel (3.9 f.e.)
> 
> The are quite some granttable messages in xl dmesg, i also included these
and a "vmstat -m"
> 
This looks like a normal OOM. alloc_netdev is not able to allocate
xenvif structure, which is only a few hundred / thousand bytes (don''t
remember the exact number).

Are you using persistent grant in blkback? I think those grant table
messages come from blkback hording grant references. It also implies it
might be hording Dom0 pages. You can try to turn off persistent grant
and see if it works.

Wei.
> --
> Sander
> 
> [54807.299791] xenwatch: page allocation failure: order:4, mode:0x10c0d0
> [54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted 3.12.0-20131104 #1
> [54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS
V1.8B1 09/13/2010
> [54807.351990]  0000000000000000 ffff8800591a3828 ffffffff81a6a8ba
0000000000000006
> [54807.368857]  000000000010c0d0 ffff8800591a38b8 ffffffff81138770
0000000000000004
> [54807.385539]  000000000010c0d0 ffff8800591a38b8 ffffffff81a661de
ffff88005fd1c680
> [54807.401978] Call Trace:
> [54807.418015]  [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
> [54807.433971]  [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
> [54807.449644]  [<ffffffff81a661de>] ?
__alloc_pages_direct_compact+0x1ac/0x1be
> [54807.465164]  [<ffffffff8113bfaa>]
__alloc_pages_nodemask+0x7aa/0x9d0
> [54807.480510]  [<ffffffff810ed069>] ?
trace_hardirqs_off_caller+0xb9/0x160
> [54807.495622]  [<ffffffff81175277>] alloc_pages_current+0xb7/0x180
> [54807.510530]  [<ffffffff81138059>] __get_free_pages+0x9/0x40
> [54807.525185]  [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
> [54807.539538]  [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
> [54807.553814]  [<ffffffff8192ac20>] ? alloc_etherdev_mqs+0x20/0x20
> [54807.567777]  [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
> [54807.581473]  [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
> [54807.594971]  [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
> [54807.608231]  [<ffffffff81615105>] driver_probe_device+0x75/0x210
> [54807.621227]  [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
> [54807.634071]  [<ffffffff8161539b>] __device_attach+0x4b/0x60
> [54807.646626]  [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
> [54807.658918]  [<ffffffff81615058>] device_attach+0x98/0xb0
> [54807.671253]  [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
> [54807.683379]  [<ffffffff81612277>] device_add+0x3b7/0x700
> [54807.695145]  [<ffffffff8161ca1d>] ? device_pm_sleep_init+0x4d/0x80
> [54807.706824]  [<ffffffff816125d9>] device_register+0x19/0x20
> [54807.718145]  [<ffffffff814bf9b1>] xenbus_probe_node+0x141/0x170
> [54807.729256]  [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
> [54807.740091]  [<ffffffff814bfbb0>] xenbus_dev_changed+0x1d0/0x1e0
> [54807.750811]  [<ffffffff814bff26>] backend_changed+0x16/0x20
> [54807.761256]  [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
> [54807.771371]  [<ffffffff810bc1e0>] ?
__init_waitqueue_head+0x60/0x60
> [54807.781443]  [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
> [54807.791310]  [<ffffffff810bb716>] kthread+0xd6/0xe0
> [54807.800794]  [<ffffffff81a752bb>] ? _raw_spin_unlock_irq+0x2b/0x70
> [54807.810150]  [<ffffffff810bb640>] ?
__init_kthread_worker+0x70/0x70
> [54807.819526]  [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
> [54807.828638]  [<ffffffff810bb640>] ?
__init_kthread_worker+0x70/0x70
> [54807.837482] Mem-Info:
> [54807.846011] Node 0 DMA per-cpu:
> [54807.854277] CPU    0: hi:    0, btch:   1 usd:   0
> [54807.862404] CPU    1: hi:    0, btch:   1 usd:   0
> [54807.870241] CPU    2: hi:    0, btch:   1 usd:   0
> [54807.877985] CPU    3: hi:    0, btch:   1 usd:   0
> [54807.885344] CPU    4: hi:    0, btch:   1 usd:   0
> [54807.892389] CPU    5: hi:    0, btch:   1 usd:   0
> [54807.899102] Node 0 DMA32 per-cpu:
> [54807.905665] CPU    0: hi:  186, btch:  31 usd:  68
> [54807.911985] CPU    1: hi:  186, btch:  31 usd:   6
> [54807.917964] CPU    2: hi:  186, btch:  31 usd: 149
> [54807.923666] CPU    3: hi:  186, btch:  31 usd:  82
> [54807.929031] CPU    4: hi:  186, btch:  31 usd: 169
> [54807.934293] CPU    5: hi:  186, btch:  31 usd:   0
> [54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
> [54807.939207]  active_file:70321 inactive_file:177850 isolated_file:0
> [54807.939207]  unevictable:562 dirty:34 writeback:0 unstable:0
> [54807.939207]  free:31143 slab_reclaimable:21805 slab_unreclaimable:12717
> [54807.939207]  mapped:3344 shmem:276 pagetables:1211 bounce:0
> [54807.939207]  free_cma:0
> [54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB
active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB
unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB
managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB
slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB
pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
> [54807.981676] lowmem_reserve[]: 0 1395 1395 1395
> [54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB high:7120kB
active_anon:51028kB inactive_anon:35772kB active_file:281280kB
inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB
present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB
mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB
kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
> [54808.011107] lowmem_reserve[]: 0 0 0 0
> [54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM) 14*32kB
(UEM) 8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB 1*2048kB (R)
0*4096kB = 5644kB
> [54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB (EMR)
6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB 0*4096kB =
118608kB
> [54808.044303] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0
hugepages_size=2048kB
> [54808.051507] 249203 total pagecache pages
> [54808.058647] 201 pages in swap cache
> [54808.065986] Swap cache stats: add 2902, delete 2701, find 62055/62191
> [54808.073216] Free swap  = 2092752kB
> [54808.080397] Total swap = 2097148kB
> [54808.093896] 393215 pages RAM
> [54808.101004] 30940 pages reserved
> [54808.108168] 339108 pages shared
> [54808.115202] 258777 pages non-shared
> [54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for
vif16.0
> [54808.129391] vif vif-16-0: 12 creating interface
> 
> 
> ~# vmstat -m
> Cache                       Num  Total   Size  Pages
> ext4_groupinfo_4k         24426  24426    216     18
> ceph_osd_request              0      0    960     17
> xt_hashlimit                  0      0    136     30
> nf_conntrack_ffffffff822d2900   1950   2100    400     20
> nf_conntrack_expect           0      0    288     28
> dm_snap_pending_exception   2496   2496    104     39
> kcopyd_job                   91    133   4176      7
> dm_rq_target_io               0      0    416     19
> search                        0      0    616     26
> blkif_cache                 108    108    864     18
> cfq_io_cq                   204    204    120     34
> cfq_queue                   170    170    232     17
> bsg_cmd                       0      0    312     26
> ceph_cap                      0      0    128     32
> ceph_inode_info               0      0   2176     15
> gfs2_mblk                     0      0    432     18
> gfs2_bufdata                  0      0     88     46
> gfs2_inode                    0      0   1280     25
> gfs2_glock(aspace)            0      0    960     17
> gfs2_glock                    0      0    592     27
> btrfs_delayed_data_ref        0      0     96     42
> btrfs_delayed_ref_head        0      0    232     17
> btrfs_delayed_node            0      0    392     20
> btrfs_ordered_extent          0      0    472     17
> btrfs_extent_buffer           0      0    568     28
> btrfs_delalloc_work           0      0    184     22
> btrfs_path                    0      0    144     28
> btrfs_transaction             0      0    600     27
> btrfs_trans_handle            0      0    160     25
> btrfs_inode                   0      0   1944     16
> fuse_request                102    102    480     17
> fuse_inode                  336    476   1152     28
> ntfs_big_inode_cache          0      0   1664     19
> ntfs_inode_cache              0      0    696     23
> cifs_small_rq                36     36    448     18
> cifs_request                  4      4  16512      1
> cifs_inode_cache              0      0   1112     29
> isofs_inode_cache             0      0    960     17
> fat_inode_cache               0      0   1168     28
> fat_cache                     0      0     40    102
> hugetlbfs_inode_cache        16     16    976     16
> jbd2_transaction_s          150    150    320     25
> jbd2_journal_handle         306    306     80     51
> journal_handle                0      0     56     73
> journal_head               1518   1800    112     36
> revoke_table               1536   1536     16    256
> revoke_record               768    768     32    128
> ext4_inode_cache          30702  30952   1704     19
> ext4_free_data              768    768     64     64
> ext4_allocation_context     180    180    136     30
> ext4_prealloc_space         156    156    152     26
> ext4_io_end                 336    336     72     56
> ext4_extent_status         7361   7446     40    102
> ext3_inode_cache              0      0   1312     24
> ext3_xattr                 3726   3956     88     46
> dquot                         0      0    384     21
> kioctx                        0      0    896     18
> pid_namespace                 0      0   2208     14
> posix_timers_cache          108    108    296     27
> UNIX                        198    198   1472     22
> Cache                       Num  Total   Size  Pages
> UDP-Lite                      0      0   1280     25
> ip_fib_trie                 438    438     56     73
> PING                          0      0   1216     26
> UDP                         150    150   1280     25
> tw_sock_TCP                 399    399    192     21
> TCP                         104    104   2368     13
> fscache_cookie_jar            0      0    192     21
> sgpool-128                   54     78   5120      6
> sgpool-64                    72     72   2560     12
> sgpool-32                   325    325   1280     25
> sgpool-16                   277    350    640     25
> blkdev_integrity              0      0    112     36
> blkdev_queue                165    165   2872     11
> blkdev_requests             315    315    376     21
> fsnotify_event_holder         0      0     24    170
> sock_inode_cache            272    272    960     17
> file_lock_cache             102    102    240     17
> shmem_inode_cache          2019   2072   1144     28
> Acpi-State                  255    255     80     51
> Acpi-Namespace             1428   1428     40    102
> task_delay_info            1704   1704    168     24
> taskstats                   144    144    328     24
> proc_inode_cache           1632   1664    976     16
> sigqueue                    250    250    160     25
> bdev_cache                  192    192   1344     24
> sysfs_dir_cache           23817  23968    144     28
> filp                       1807   2400    320     25
> inode_cache                1861   2244    912     17
> dentry                    27252  29440    248     16
> buffer_head               67283 102063    104     39
> vm_area_struct             4527   4708    184     22
> mm_struct                   308    308   1152     28
> files_cache                 253    253    704     23
> signal_cache                667    667   1408     23
> sighand_cache               407    490   2240     14
> task_struct                 430    442   4240      7
> anon_vma                   4676   4676    144     28
> shared_policy_node          850    850     48     85
> numa_policy                3528   3528     72     56
> radix_tree_node            7622   8050    568     28
> idr_layer_cache             240    240   2112     15
> dma-kmalloc-8192              0      0   8192      4
> dma-kmalloc-4096              0      0   4096      8
> dma-kmalloc-2048              0      0   2048     16
> dma-kmalloc-1024              0      0   1024     16
> dma-kmalloc-512               0      0    512     16
> dma-kmalloc-256               0      0    256     16
> dma-kmalloc-128               0      0    128     32
> dma-kmalloc-64                0      0     64     64
> dma-kmalloc-32                0      0     32    128
> dma-kmalloc-16                0      0     16    256
> dma-kmalloc-8                 0      0      8    512
> dma-kmalloc-192               0      0    192     21
> dma-kmalloc-96                0      0     96     42
> kmalloc-8192                 32     32   8192      4
> kmalloc-4096                835    916   4096      8
> kmalloc-2048                773    832   2048     16
> kmalloc-1024               2335   2336   1024     16
> kmalloc-512                1488   1536    512     16
> kmalloc-256                2535   2784    256     16
> Cache                       Num  Total   Size  Pages
> kmalloc-192               69883  70707    192     21
> kmalloc-128                2603   4992    128     32
> kmalloc-96                 5628   5628     96     42
> kmalloc-64                41984  41984     64     64
> kmalloc-32               154792 156288     32    128
> kmalloc-16                 6343   7936     16    256
> kmalloc-8                  8192   8192      8    512
> kmem_cache_node             256    256    128     32
> kmem_cache                  176    176    256     16
> 
> 
> (XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1) grant
table from (4) to (5) frames.
> (XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack size to
8 frames
> (XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1) grant
table from (7) to (8) frames.
> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to
9 frames
> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack size to
10 frames
> (XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1) grant
table from (8) to (9) frames.
> (XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1) grant
table from (9) to (10) frames.
> (XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1) grant
table from (10) to (11) frames.
> (XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack size to
11 frames
> (XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack size to
12 frames
> (XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack size to
13 frames
> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack size to
14 frames
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to
15 frames
> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack size to
16 frames
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to
17 frames
> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack size to
18 frames
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack size to
19 frames
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack size to
20 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to
21 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack size to
22 frames
> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to
23 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to
24 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to
25 frames
> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack size to
26 frames
> (XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack size to
27 frames
> (XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack size to
28 frames
> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack size to
29 frames
> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack size to
30 frames
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant
table from (4) to (5) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant
table from (5) to (6) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6) grant
table from (6) to (7) frames.
> (XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack size to
31 frames
> (XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m table =
0x52d3c5

Sander Eikelenboom

2013-Nov-19 13:40 UTC

head link

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Tuesday, November 19, 2013, 2:23:35 PM, you wrote:
> On Tue, Nov 19, 2013 at 02:02:57PM +0100, Sander Eikelenboom wrote:
>> Hi Wei,
>> 
>> I ran into the following problem when trying to boot another guest
after less than a day of uptime.
>> (the system started 15 guests at boot already which went fine). dom0 is
allocated a fixed 1536M.
>> 
>> Both host as pv guests run the same kernel, some hvm''s run a
slightly older kernel (3.9 f.e.)
>> 
>> The are quite some granttable messages in xl dmesg, i also included
these and a "vmstat -m"
>> 
> This looks like a normal OOM. alloc_netdev is not able to allocate
> xenvif structure, which is only a few hundred / thousand bytes
(don''t
> remember the exact number).
> Are you using persistent grant in blkback? I think those grant table
> messages come from blkback hording grant references. It also implies it
> might be hording Dom0 pages. You can try to turn off persistent grant
> and see if it works.
Yes it does use persistent grants, at least blkback does, i don''t see
any reference of netback ..

# dmesg | grep persistent
[  130.385948] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi)
persistent grants
[  130.401092] xen-blkback:ring-ref 9, event-channel 18, protocol 1 (x86_64-abi)
persistent grants
[  136.414469] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  142.772308] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi)
persistent grants
[  148.152987] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  154.047275] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  160.033898] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  166.474991] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi)
persistent grants
[  172.111850] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  180.454091] xen-blkback:ring-ref 8, event-channel 17, protocol 1 (x86_64-abi)
persistent grants
[  186.935862] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  193.496961] xen-blkback:ring-ref 9, event-channel 11, protocol 1 (x86_64-abi)
persistent grants
[  200.179170] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  206.476770] xen-blkback:ring-ref 8, event-channel 10, protocol 1 (x86_64-abi)
persistent grants
[  259.796801] xen-blkback:ring-ref 8, event-channel 32, protocol 1 (x86_64-abi)
persistent grants
[  259.811314] xen-blkback:ring-ref 9, event-channel 33, protocol 1 (x86_64-abi)
persistent grants
[  264.968420] xen-blkback:ring-ref 8, event-channel 22, protocol 1 (x86_64-abi)
persistent grants


hmm naive as i am i thought that would limit the mem usage by re-using the
grants (and thus pages).
Why does it keep increasing the number of grants ?
Doesn''t the device rings have a max size, so once that''s
reached it should stop allocating new and only reuse it persitent grants ?

Also it must be a special memtype/size ? Dom0 was only using about 400M of
it''s 1536M.
> Wei.
>> --
>> Sander
>> 
>> [54807.299791] xenwatch: page allocation failure: order:4,
mode:0x10c0d0
>> [54807.317520] CPU: 5 PID: 54 Comm: xenwatch Not tainted
3.12.0-20131104 #1
>> [54807.334747] Hardware name: MSI MS-7640/890FXA-GD70 (MS-7640)  , BIOS
V1.8B1 09/13/2010
>> [54807.351990]  0000000000000000 ffff8800591a3828 ffffffff81a6a8ba
0000000000000006
>> [54807.368857]  000000000010c0d0 ffff8800591a38b8 ffffffff81138770
0000000000000004
>> [54807.385539]  000000000010c0d0 ffff8800591a38b8 ffffffff81a661de
ffff88005fd1c680
>> [54807.401978] Call Trace:
>> [54807.418015]  [<ffffffff81a6a8ba>] dump_stack+0x4f/0x84
>> [54807.433971]  [<ffffffff81138770>] warn_alloc_failed+0xf0/0x140
>> [54807.449644]  [<ffffffff81a661de>] ?
__alloc_pages_direct_compact+0x1ac/0x1be
>> [54807.465164]  [<ffffffff8113bfaa>]
__alloc_pages_nodemask+0x7aa/0x9d0
>> [54807.480510]  [<ffffffff810ed069>] ?
trace_hardirqs_off_caller+0xb9/0x160
>> [54807.495622]  [<ffffffff81175277>]
alloc_pages_current+0xb7/0x180
>> [54807.510530]  [<ffffffff81138059>] __get_free_pages+0x9/0x40
>> [54807.525185]  [<ffffffff8117cbdc>] __kmalloc+0x19c/0x1c0
>> [54807.539538]  [<ffffffff8190e9b4>] alloc_netdev_mqs+0x64/0x340
>> [54807.553814]  [<ffffffff8192ac20>] ?
alloc_etherdev_mqs+0x20/0x20
>> [54807.567777]  [<ffffffff816dc3e4>] xenvif_alloc+0x64/0x2c0
>> [54807.581473]  [<ffffffff816dbc57>] netback_probe+0x287/0x2d0
>> [54807.594971]  [<ffffffff814bfe46>] xenbus_dev_probe+0x66/0x110
>> [54807.608231]  [<ffffffff81615105>]
driver_probe_device+0x75/0x210
>> [54807.621227]  [<ffffffff81615350>] ? __driver_attach+0xb0/0xb0
>> [54807.634071]  [<ffffffff8161539b>] __device_attach+0x4b/0x60
>> [54807.646626]  [<ffffffff8161316e>] bus_for_each_drv+0x4e/0xa0
>> [54807.658918]  [<ffffffff81615058>] device_attach+0x98/0xb0
>> [54807.671253]  [<ffffffff816144b0>] bus_probe_device+0xb0/0xe0
>> [54807.683379]  [<ffffffff81612277>] device_add+0x3b7/0x700
>> [54807.695145]  [<ffffffff8161ca1d>] ?
device_pm_sleep_init+0x4d/0x80
>> [54807.706824]  [<ffffffff816125d9>] device_register+0x19/0x20
>> [54807.718145]  [<ffffffff814bf9b1>]
xenbus_probe_node+0x141/0x170
>> [54807.729256]  [<ffffffff81613236>] ? bus_for_each_dev+0x76/0xa0
>> [54807.740091]  [<ffffffff814bfbb0>]
xenbus_dev_changed+0x1d0/0x1e0
>> [54807.750811]  [<ffffffff814bff26>] backend_changed+0x16/0x20
>> [54807.761256]  [<ffffffff814bdf3e>] xenwatch_thread+0x4e/0x140
>> [54807.771371]  [<ffffffff810bc1e0>] ?
__init_waitqueue_head+0x60/0x60
>> [54807.781443]  [<ffffffff814bdef0>] ? xs_watch+0x60/0x60
>> [54807.791310]  [<ffffffff810bb716>] kthread+0xd6/0xe0
>> [54807.800794]  [<ffffffff81a752bb>] ?
_raw_spin_unlock_irq+0x2b/0x70
>> [54807.810150]  [<ffffffff810bb640>] ?
__init_kthread_worker+0x70/0x70
>> [54807.819526]  [<ffffffff81a762cc>] ret_from_fork+0x7c/0xb0
>> [54807.828638]  [<ffffffff810bb640>] ?
__init_kthread_worker+0x70/0x70
>> [54807.837482] Mem-Info:
>> [54807.846011] Node 0 DMA per-cpu:
>> [54807.854277] CPU    0: hi:    0, btch:   1 usd:   0
>> [54807.862404] CPU    1: hi:    0, btch:   1 usd:   0
>> [54807.870241] CPU    2: hi:    0, btch:   1 usd:   0
>> [54807.877985] CPU    3: hi:    0, btch:   1 usd:   0
>> [54807.885344] CPU    4: hi:    0, btch:   1 usd:   0
>> [54807.892389] CPU    5: hi:    0, btch:   1 usd:   0
>> [54807.899102] Node 0 DMA32 per-cpu:
>> [54807.905665] CPU    0: hi:  186, btch:  31 usd:  68
>> [54807.911985] CPU    1: hi:  186, btch:  31 usd:   6
>> [54807.917964] CPU    2: hi:  186, btch:  31 usd: 149
>> [54807.923666] CPU    3: hi:  186, btch:  31 usd:  82
>> [54807.929031] CPU    4: hi:  186, btch:  31 usd: 169
>> [54807.934293] CPU    5: hi:  186, btch:  31 usd:   0
>> [54807.939207] active_anon:12850 inactive_anon:8944 isolated_anon:0
>> [54807.939207]  active_file:70321 inactive_file:177850 isolated_file:0
>> [54807.939207]  unevictable:562 dirty:34 writeback:0 unstable:0
>> [54807.939207]  free:31143 slab_reclaimable:21805
slab_unreclaimable:12717
>> [54807.939207]  mapped:3344 shmem:276 pagetables:1211 bounce:0
>> [54807.939207]  free_cma:0
>> [54807.966465] Node 0 DMA free:5644kB min:52kB low:64kB high:76kB
active_anon:0kB inactive_anon:4kB active_file:4kB inactive_file:0kB
unevictable:24kB isolated(anon):0kB isolated(file):0kB present:15968kB
managed:15884kB mlocked:24kB dirty:0kB writeback:0kB mapped:24kB shmem:0kB
slab_reclaimable:8992kB slab_unreclaimable:656kB kernel_stack:16kB
pagetables:56kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
>> [54807.981676] lowmem_reserve[]: 0 1395 1395 1395
>> [54807.987015] Node 0 DMA32 free:118928kB min:4748kB low:5932kB
high:7120kB active_anon:51028kB inactive_anon:35772kB active_file:281280kB
inactive_file:711400kB unevictable:2224kB isolated(anon):0kB isolated(file):0kB
present:1556480kB managed:1433216kB mlocked:2224kB dirty:136kB writeback:0kB
mapped:13168kB shmem:1104kB slab_reclaimable:78228kB slab_unreclaimable:50212kB
kernel_stack:3256kB pagetables:4788kB unstable:0kB bounce:0kB free_cma:0kB
writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
>> [54808.011107] lowmem_reserve[]: 0 0 0 0
>> [54808.017244] Node 0 DMA: 19*4kB (UEM) 18*8kB (UEM) 15*16kB (UEM)
14*32kB (UEM) 8*64kB (UEM) 7*128kB (UEM) 3*256kB (U) 1*512kB (U) 0*1024kB
1*2048kB (R) 0*4096kB = 5644kB
>> [54808.030315] Node 0 DMA32: 11490*4kB (UEM) 6743*8kB (UEM) 1133*16kB
(EMR) 6*32kB (MR) 0*64kB 1*128kB (R) 1*256kB (R) 0*512kB 0*1024kB 0*2048kB
0*4096kB = 118608kB
>> [54808.044303] Node 0 hugepages_total=0 hugepages_free=0
hugepages_surp=0 hugepages_size=2048kB
>> [54808.051507] 249203 total pagecache pages
>> [54808.058647] 201 pages in swap cache
>> [54808.065986] Swap cache stats: add 2902, delete 2701, find
62055/62191
>> [54808.073216] Free swap  = 2092752kB
>> [54808.080397] Total swap = 2097148kB
>> [54808.093896] 393215 pages RAM
>> [54808.101004] 30940 pages reserved
>> [54808.108168] 339108 pages shared
>> [54808.115202] 258777 pages non-shared
>> [54808.122235] xen_netback:xenvif_alloc: Could not allocate netdev for
vif16.0
>> [54808.129391] vif vif-16-0: 12 creating interface
>> 
>> 
>> ~# vmstat -m
>> Cache                       Num  Total   Size  Pages
>> ext4_groupinfo_4k         24426  24426    216     18
>> ceph_osd_request              0      0    960     17
>> xt_hashlimit                  0      0    136     30
>> nf_conntrack_ffffffff822d2900   1950   2100    400     20
>> nf_conntrack_expect           0      0    288     28
>> dm_snap_pending_exception   2496   2496    104     39
>> kcopyd_job                   91    133   4176      7
>> dm_rq_target_io               0      0    416     19
>> search                        0      0    616     26
>> blkif_cache                 108    108    864     18
>> cfq_io_cq                   204    204    120     34
>> cfq_queue                   170    170    232     17
>> bsg_cmd                       0      0    312     26
>> ceph_cap                      0      0    128     32
>> ceph_inode_info               0      0   2176     15
>> gfs2_mblk                     0      0    432     18
>> gfs2_bufdata                  0      0     88     46
>> gfs2_inode                    0      0   1280     25
>> gfs2_glock(aspace)            0      0    960     17
>> gfs2_glock                    0      0    592     27
>> btrfs_delayed_data_ref        0      0     96     42
>> btrfs_delayed_ref_head        0      0    232     17
>> btrfs_delayed_node            0      0    392     20
>> btrfs_ordered_extent          0      0    472     17
>> btrfs_extent_buffer           0      0    568     28
>> btrfs_delalloc_work           0      0    184     22
>> btrfs_path                    0      0    144     28
>> btrfs_transaction             0      0    600     27
>> btrfs_trans_handle            0      0    160     25
>> btrfs_inode                   0      0   1944     16
>> fuse_request                102    102    480     17
>> fuse_inode                  336    476   1152     28
>> ntfs_big_inode_cache          0      0   1664     19
>> ntfs_inode_cache              0      0    696     23
>> cifs_small_rq                36     36    448     18
>> cifs_request                  4      4  16512      1
>> cifs_inode_cache              0      0   1112     29
>> isofs_inode_cache             0      0    960     17
>> fat_inode_cache               0      0   1168     28
>> fat_cache                     0      0     40    102
>> hugetlbfs_inode_cache        16     16    976     16
>> jbd2_transaction_s          150    150    320     25
>> jbd2_journal_handle         306    306     80     51
>> journal_handle                0      0     56     73
>> journal_head               1518   1800    112     36
>> revoke_table               1536   1536     16    256
>> revoke_record               768    768     32    128
>> ext4_inode_cache          30702  30952   1704     19
>> ext4_free_data              768    768     64     64
>> ext4_allocation_context     180    180    136     30
>> ext4_prealloc_space         156    156    152     26
>> ext4_io_end                 336    336     72     56
>> ext4_extent_status         7361   7446     40    102
>> ext3_inode_cache              0      0   1312     24
>> ext3_xattr                 3726   3956     88     46
>> dquot                         0      0    384     21
>> kioctx                        0      0    896     18
>> pid_namespace                 0      0   2208     14
>> posix_timers_cache          108    108    296     27
>> UNIX                        198    198   1472     22
>> Cache                       Num  Total   Size  Pages
>> UDP-Lite                      0      0   1280     25
>> ip_fib_trie                 438    438     56     73
>> PING                          0      0   1216     26
>> UDP                         150    150   1280     25
>> tw_sock_TCP                 399    399    192     21
>> TCP                         104    104   2368     13
>> fscache_cookie_jar            0      0    192     21
>> sgpool-128                   54     78   5120      6
>> sgpool-64                    72     72   2560     12
>> sgpool-32                   325    325   1280     25
>> sgpool-16                   277    350    640     25
>> blkdev_integrity              0      0    112     36
>> blkdev_queue                165    165   2872     11
>> blkdev_requests             315    315    376     21
>> fsnotify_event_holder         0      0     24    170
>> sock_inode_cache            272    272    960     17
>> file_lock_cache             102    102    240     17
>> shmem_inode_cache          2019   2072   1144     28
>> Acpi-State                  255    255     80     51
>> Acpi-Namespace             1428   1428     40    102
>> task_delay_info            1704   1704    168     24
>> taskstats                   144    144    328     24
>> proc_inode_cache           1632   1664    976     16
>> sigqueue                    250    250    160     25
>> bdev_cache                  192    192   1344     24
>> sysfs_dir_cache           23817  23968    144     28
>> filp                       1807   2400    320     25
>> inode_cache                1861   2244    912     17
>> dentry                    27252  29440    248     16
>> buffer_head               67283 102063    104     39
>> vm_area_struct             4527   4708    184     22
>> mm_struct                   308    308   1152     28
>> files_cache                 253    253    704     23
>> signal_cache                667    667   1408     23
>> sighand_cache               407    490   2240     14
>> task_struct                 430    442   4240      7
>> anon_vma                   4676   4676    144     28
>> shared_policy_node          850    850     48     85
>> numa_policy                3528   3528     72     56
>> radix_tree_node            7622   8050    568     28
>> idr_layer_cache             240    240   2112     15
>> dma-kmalloc-8192              0      0   8192      4
>> dma-kmalloc-4096              0      0   4096      8
>> dma-kmalloc-2048              0      0   2048     16
>> dma-kmalloc-1024              0      0   1024     16
>> dma-kmalloc-512               0      0    512     16
>> dma-kmalloc-256               0      0    256     16
>> dma-kmalloc-128               0      0    128     32
>> dma-kmalloc-64                0      0     64     64
>> dma-kmalloc-32                0      0     32    128
>> dma-kmalloc-16                0      0     16    256
>> dma-kmalloc-8                 0      0      8    512
>> dma-kmalloc-192               0      0    192     21
>> dma-kmalloc-96                0      0     96     42
>> kmalloc-8192                 32     32   8192      4
>> kmalloc-4096                835    916   4096      8
>> kmalloc-2048                773    832   2048     16
>> kmalloc-1024               2335   2336   1024     16
>> kmalloc-512                1488   1536    512     16
>> kmalloc-256                2535   2784    256     16
>> Cache                       Num  Total   Size  Pages
>> kmalloc-192               69883  70707    192     21
>> kmalloc-128                2603   4992    128     32
>> kmalloc-96                 5628   5628     96     42
>> kmalloc-64                41984  41984     64     64
>> kmalloc-32               154792 156288     32    128
>> kmalloc-16                 6343   7936     16    256
>> kmalloc-8                  8192   8192      8    512
>> kmem_cache_node             256    256    128     32
>> kmem_cache                  176    176    256     16
>> 
>> 
>> (XEN) [2013-11-18 21:44:32] grant_table.c:1249:d1 Expanding dom (1)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-18 21:44:32] grant_table.c:289:d0 Increased maptrack
size to 8 frames
>> (XEN) [2013-11-18 21:49:28] grant_table.c:1249:d1 Expanding dom (1)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:1249:d1 Expanding dom (1)
grant table from (7) to (8) frames.
>> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack
size to 9 frames
>> (XEN) [2013-11-19 00:00:26] grant_table.c:289:d0 Increased maptrack
size to 10 frames
>> (XEN) [2013-11-19 00:00:42] grant_table.c:1249:d1 Expanding dom (1)
grant table from (8) to (9) frames.
>> (XEN) [2013-11-19 00:01:02] grant_table.c:1249:d1 Expanding dom (1)
grant table from (9) to (10) frames.
>> (XEN) [2013-11-19 00:09:27] grant_table.c:1249:d1 Expanding dom (1)
grant table from (10) to (11) frames.
>> (XEN) [2013-11-19 00:09:27] grant_table.c:289:d0 Increased maptrack
size to 11 frames
>> (XEN) [2013-11-19 04:15:26] grant_table.c:289:d0 Increased maptrack
size to 12 frames
>> (XEN) [2013-11-19 04:15:28] grant_table.c:1249:d12 Expanding dom (12)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:40] grant_table.c:289:d0 Increased maptrack
size to 13 frames
>> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:47] grant_table.c:1249:d10 Expanding dom (10)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:47] grant_table.c:289:d0 Increased maptrack
size to 14 frames
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:1249:d5 Expanding dom (5)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack
size to 15 frames
>> (XEN) [2013-11-19 04:15:52] grant_table.c:289:d0 Increased maptrack
size to 16 frames
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:1249:d8 Expanding dom (8)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack
size to 17 frames
>> (XEN) [2013-11-19 04:15:54] grant_table.c:289:d0 Increased maptrack
size to 18 frames
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:1249:d2 Expanding dom (2)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:56] grant_table.c:289:d0 Increased maptrack
size to 19 frames
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:1249:d3 Expanding dom (3)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:15:57] grant_table.c:289:d0 Increased maptrack
size to 20 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack
size to 21 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:00] grant_table.c:289:d0 Increased maptrack
size to 22 frames
>> (XEN) [2013-11-19 04:16:00] grant_table.c:1249:d13 Expanding dom (13)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d9 Expanding dom (9)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack
size to 23 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack
size to 24 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:1249:d4 Expanding dom (4)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack
size to 25 frames
>> (XEN) [2013-11-19 04:16:03] grant_table.c:289:d0 Increased maptrack
size to 26 frames
>> (XEN) [2013-11-19 04:16:06] grant_table.c:1249:d10 Expanding dom (10)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:1249:d12 Expanding dom (12)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 04:16:22] grant_table.c:289:d0 Increased maptrack
size to 27 frames
>> (XEN) [2013-11-19 04:16:23] grant_table.c:289:d0 Increased maptrack
size to 28 frames
>> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 04:16:24] grant_table.c:1249:d11 Expanding dom (11)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:24] grant_table.c:289:d0 Increased maptrack
size to 29 frames
>> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 04:16:27] grant_table.c:1249:d7 Expanding dom (7)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 05:15:04] grant_table.c:289:d0 Increased maptrack
size to 30 frames
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6)
grant table from (4) to (5) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6)
grant table from (5) to (6) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:1249:d6 Expanding dom (6)
grant table from (6) to (7) frames.
>> (XEN) [2013-11-19 05:15:05] grant_table.c:289:d0 Increased maptrack
size to 31 frames
>> (XEN) [2013-11-19 12:49:37] AMD-Vi: Share p2m table with iommu: p2m
table = 0x52d3c5

David Vrabel

2013-Nov-19 13:41 UTC

head link

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

On 19/11/13 13:02, Sander Eikelenboom wrote:> Hi Wei,
> 
> I ran into the following problem when trying to boot another guest after
less than a day of uptime.
> (the system started 15 guests at boot already which went fine). dom0 is
allocated a fixed 1536M.
> 
> Both host as pv guests run the same kernel, some hvm''s run a
slightly older kernel (3.9 f.e.)
> 
> The are quite some granttable messages in xl dmesg, i also included these
and a "vmstat -m"
Thread-per-VIF made the VIF''s net device structure enormous.

This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
net_device allocation to vmalloc()).

David

Wei Liu

2013-Nov-19 13:51 UTC

head link

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

On Tue, Nov 19, 2013 at 01:41:12PM +0000, David Vrabel
wrote:> On 19/11/13 13:02, Sander Eikelenboom wrote:
> > Hi Wei,
> > 
> > I ran into the following problem when trying to boot another guest
after less than a day of uptime.
> > (the system started 15 guests at boot already which went fine). dom0
is allocated a fixed 1536M.
> > 
> > Both host as pv guests run the same kernel, some hvm''s run a
slightly older kernel (3.9 f.e.)
> > 
> > The are quite some granttable messages in xl dmesg, i also included
these and a "vmstat -m"
> 
> Thread-per-VIF made the VIF''s net device structure enormous.
> 
> This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
> net_device allocation to vmalloc()).
> 
Thanks David.

Sander, you can also try to cherry-pick the above patch to see if it
works.

Wei.
> David

Sander Eikelenboom

2013-Nov-19 14:17 UTC

head link

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Tuesday, November 19, 2013, 2:51:54 PM, you wrote:
> On Tue, Nov 19, 2013 at 01:41:12PM +0000, David Vrabel wrote:
>> On 19/11/13 13:02, Sander Eikelenboom wrote:
>> > Hi Wei,
>> > 
>> > I ran into the following problem when trying to boot another guest
after less than a day of uptime.
>> > (the system started 15 guests at boot already which went fine).
dom0 is allocated a fixed 1536M.
>> > 
>> > Both host as pv guests run the same kernel, some hvm''s
run a slightly older kernel (3.9 f.e.)
>> > 
>> > The are quite some granttable messages in xl dmesg, i also
included these and a "vmstat -m"
>> 
>> Thread-per-VIF made the VIF''s net device structure enormous.
>> 
>> This is fixed by 74d332c13b2148ae934ea94dac1745ae92efe8e5 (net: extend
>> net_device allocation to vmalloc()).
>> 
Ah thanks, the merge window had some other surprises so i had to revert back to
3.12,
will try the cherry-pick .. thx for the pointer !

--
Sander

> Thanks David.
> Sander, you can also try to cherry-pick the above patch to see if it
> works.
> Wei.
>> David

Seemingly Similar Threads

Search for more seemingly similar threads

Xen devel - Nov 2013 - xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Re: xenwatch: page allocation failure: order:4, mode:0x10c0d0 xen_netback:xenvif_alloc: Could not allocate netdev for vif16.0

Seemingly Similar Threads