$ g. {; E9 O! U) ]# ~9 _7 Q. ]Jan 14 04:06:25 localhost kernel: [9821120.248545] ------------[ cut here ]------------( m* ^' @' { `/ u) z
Jan 14 04:06:25 localhost kernel: [9821120.248573] WARNING: CPU: 2 PID: 13942 at fs/dcache.c:362 d_shrink_del+0x2c/0x6e()
! e# }) y& o/ M* u, D* N( c* O9 gJan 14 04:06:25 localhost kernel: [9821120.248584] Modules linked in: deflate ctr twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common camellia_generic serpent_sse2_x86_64 ablk_helper cryptd glue_helper lrw serpent_generic xts gf128mul blowfish_ge
4 D& [6 n1 Q& K, c0 N7 ?( sneric blowfish_x86_64 blowfish_common cast5_generic cast_common des_generic cbc xcbc rmd160 sha512_generic sha256_generic crypto_null xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo 8021q mrp garp stp llc cls_u32 sch_htb xt_nat xt_LOG xt_comm
# J" f, ?# `8 v! W* @2 Hent nf_conntrack_netlink iptable_nat nf_nat_ipv4 ip6table_filter ip6table_raw ip6_tables iptable_filter nf_conntrack_ipv4 nf_defrag_ipv4 xt_CT nfnetlink_cthelper nfnetlink iptable_raw nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_h323 nf_conntrack_h323 nf_) h3 q9 E* }) S7 v
nat_sip nf_conntrack_sip nf_nat_proto_gre nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_ondemand cpufreq_conservative ipv6 fuse i2c_piix4 i2: Q5 \" g6 D; a Z
Jan 14 04:06:25 localhost kernel: [9821120.249030] CPU: 2 PID: 13942 Comm: sh Not tainted 3.13.11-1-amd64-vyos #1, i% O v0 Q: c; G' c0 d h
Jan 14 04:06:25 localhost kernel: [9821120.249043] Hardware name: Alcatel-Lucent ACCS NFV r1.0 HA, BIOS 0.5.1 01/01/2011% T9 _) X# g4 j
Jan 14 04:06:25 localhost kernel: [9821120.249054] 0000000000000000 ffffffff814f58a3 ffffffff813eb0b1 0000000000000000
( x; X3 ?9 k# m. b& n7 NJan 14 04:06:25 localhost kernel: [9821120.249125] ffffffff8104a614 ffff880215609380 ffffffff81133665 ffffffff813edfd2
8 C6 R0 @$ K; o8 OJan 14 04:06:25 localhost kernel: [9821120.249143] ffff880080cc66c0 ffff880080cc6718 ffff880231b1dce0 ffff880231b1dca0
# x/ `5 a x* Z. V# W' uJan 14 04:06:25 localhost kernel: [9821120.249161] Call Trace:
0 Y. X" q6 R$ z& v1 x8 I: m- yJan 14 04:06:25 localhost kernel: [9821120.249183] [<ffffffff813eb0b1>] ? dump_stack+0x41/0x58
3 C2 W4 c. p# q+ R8 q7 y& S/ aJan 14 04:06:25 localhost kernel: [9821120.249208] [<ffffffff8104a614>] ? warn_slowpath_common+0x79/0x92
5 d1 w9 R* z: I! B" KJan 14 04:06:25 localhost kernel: [9821120.249230] [<ffffffff81133665>] ? d_shrink_del+0x2c/0x6e( F4 P7 d9 Y) }9 l5 [: @ D' F8 O! u
Jan 14 04:06:25 localhost kernel: [9821120.249250] [<ffffffff813edfd2>] ? _raw_spin_unlock+0x5/0x6
' z- w+ K/ H- t- w LJan 14 04:06:25 localhost kernel: [9821120.249273] [<ffffffff81133665>] ? d_shrink_del+0x2c/0x6e
A2 O; G/ @1 K9 BJan 14 04:06:25 localhost kernel: [9821120.249295] [<ffffffff81134667>] ? shrink_dentry_list+0x51/0xc6
6 v5 n% \7 G" N9 u {Jan 14 04:06:25 localhost kernel: [9821120.249315] [<ffffffff813edfd2>] ? _raw_spin_unlock+0x5/0x6
; r u, a T' uJan 14 04:06:25 localhost kernel: [9821120.249339] [<ffffffff811347a5>] ? shrinJan 14 03:30:34 localhost kernel: [ 0.000000] Linux version 3.13.11-1-amd64-vyos (jenkins@squeeze64devel) (gcc version 4.4.5 (Debian 4.4.5-8) ) #1 SMP Wed Aug 12 02:08:05 UTC 2015
) k, b2 P8 R1 p. J5 x" a0 z# w6 sJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: RSDP 00000000000f70d0 000014 (v00 BOCHS )
- P( F( W9 t6 V. zJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: RSDT 00000000bffff70a 00002C (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)
# O. U6 n# `$ c9 gJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: FACP 00000000bffff606 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)
$ l p. q) O4 n: u# E( KJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: DSDT 00000000bfffe040 0015C6 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
7 C- G2 q6 {! G. u0 m$ O$ `& S9 XJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: FACS 00000000bfffe000 0000408 `9 G8 g& X; _( J9 K" B: \$ k
Jan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: APIC 00000000bffff67a 000090 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)9 x( j ^6 P q8 b: R
Jan 14 03:30:34 localhost kernel: [ 0.000000] Zone ranges:
4 W% @6 w5 ^: F7 v$ VJan 14 03:30:34 localhost kernel: [ 0.000000] DMA [mem 0x00001000-0x00ffffff]
. _5 C6 U, X. N, p' s( e* Z/ MJan 14 03:30:34 localhost kernel: [ 0.000000] DMA32 [mem 0x01000000-0xffffffff]' m H4 |5 w0 E: p
Jan 14 03:30:34 localhost kernel: [ 0.000000] Normal [mem 0x100000000-0x23fffffff]
9 V! @ @2 ^* l) V) t# WJan 14 03:30:34 localhost kernel: [ 0.000000] Movable zone start for each node7 _9 u9 T6 Y; K/ V; O6 k
Jan 14 03:30:34 localhost kernel: [ 0.000000] Early memory node ranges/ [3 n$ e" K3 ?5 Q' v
Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x00001000-0x0009efff]! j' J" ~" v! G1 m
Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x00100000-0xbfff9fff]
1 ?8 d+ `8 I$ [Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x100000000-0x23fffffff]3 P$ K4 Z- U" n' i& L1 y
Jan 14 03:30:34 localhost kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2064258; b4 y' p0 v: t0 d1 }+ O
Jan 14 03:30:34 localhost kernel: [ 0.000000] Policy zone: Normal2 g7 v& p) g1 d! p+ |
Jan 14 03:30:34 localhost kernel: [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/1.1.7/vmlinuz boot=live quiet vyatta-union=/boot/1.1.7 console=ttyS0,9600 console=tty0. j, j1 f) L+ h/ ` P3 N
Jan 14 03:30:34 localhost kernel: [ 0.000000] Memory: 8167636K/8388192K available (4043K kernel code, 594K rwdata, 1652K rodata, 976K init, 700K bss, 220556K reserved)
" [& A* z4 A; Q( E/ K, [7 i! JJan 14 03:30:34 localhost kernel: [ 0.040964] ACPI: All ACPI Tables successfully acquired
3 {, c" F8 x' FJan 14 03:30:34 localhost kernel: [ 0.050000] Failed to access perfctr msr (MSR c1 is 0)
4 D# u) S \7 P$ o9 E0 h7 IJan 14 03:30:34 localhost kernel: [ 0.142534] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20131115/hwxface-580)$ m0 M" Q4 N6 ^( d2 [: B6 v9 [
Jan 14 03:30:34 localhost kernel: [ 0.142541] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20131115/hwxface-580)
, N# H7 c. n$ B" { P1 Y. SJan 14 03:30:34 localhost kernel: [ 0.142545] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S3_] (20131115/hwxface-580)
( Y2 C# ]4 a! Y# K2 u) w! UJan 14 03:30:34 localhost kernel: [ 0.142549] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S4_] (20131115/hwxface-580)
2 Z# | G* n6 y/ k- ZJan 14 03:30:34 localhost kernel: [ 0.150892] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
( B" E i$ l, X1 C: c! R6 g9 s4 ^Jan 14 03:30:34 localhost kernel: [ 0.443990] ACPI: Enabled 3 GPEs in block 00 to 0F
+ d$ w& m1 x" a- UJan 14 03:30:34 localhost kernel: [ 0.444104] SCSI subsystem initialized
4 X) s$ l# P( Q; d1 c' m& L5 }Jan 14 03:30:34 localhost kernel: [ 0.532368] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
0 Y7 p6 y$ L6 V* S' G* WJan 14 03:30:34 localhost kernel: [ 1.028215] type=2000 audit(1578972622.313:1): initialized: ?2 i( O1 u2 d5 `
Jan 14 03:30:34 localhost kernel: [ 1.029615] bounce pool size: 64 pages
" I7 w1 P, S7 n' z& R& DJan 14 03:30:34 localhost kernel: [ 1.138358] Key type dns_resolver registered0 B) c% Y6 W/ G; E J6 }; {
Jan 14 03:30:34 localhost kernel: [ 1.244013] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
9 s6 a" ~0 n% J! v }) I8 qJan 14 03:30:34 localhost kernel: [ 1.357808] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10 D, H1 Q/ p& ]8 N# K5 v8 y$ N6 K
Jan 14 03:30:34 localhost kernel: [ 2.040051] raid6: sse2x1 4639 MB/s. i5 {3 D# `* V6 _: V7 x$ `' ^
Jan 14 03:30:34 localhost kernel: [ 2.210027] raid6: sse2x2 6362 MB/s, a" T, x! T" Q0 q; }) K
Jan 14 03:30:34 localhost kernel: [ 2.380032] raid6: sse2x4 7357 MB/s
( i$ k: Z3 T3 d8 D: Z) u0 ]0 @* ]& eJan 14 03:30:34 localhost kernel: [ 2.380037] raid6: using algorithm sse2x4 (7357 MB/s), C& [# v! G/ o1 b& Z
Jan 14 03:30:34 localhost kernel: [ 2.380040] raid6: using intx1 recovery algorithm" G. [# M! {1 f! y
Jan 14 03:30:34 localhost kernel: [ 8.178340] EXT4-fs (vda1): warning: checktime reached, running e2fsck is recommended* @% ~3 F$ r5 i
Jan 14 03:30:34 localhost kernel: [ 9.087041] random: debconf-communi urandom read with 25 bits of entropy available
6 W; v5 w' k+ m3 ZJan 14 03:30:34 localhost kernel: [ 10.625650] platform microcode: Direct firmware load failed with error -27 l9 B& ?' h" x$ \4 T# ?1 M
Jan 14 03:30:34 localhost kernel: [ 10.625654] platform microcode: Falling back to user helper7 d% J8 ]. h* L
Jan 14 03:30:34 localhost kernel: [ 10.652304] platform microcode: Direct firmware load failed with error -2
$ |7 f9 ]+ S$ g2 w3 `6 U( AJan 14 03:30:34 localhost kernel: [ 10.652309] platform microcode: Falling back to user helper9 D5 o& r- m) P7 U$ j$ H
Jan 14 03:30:34 localhost kernel: [ 10.661767] platform microcode: Direct firmware load failed with error -2
2 J4 R/ f4 @( y5 P/ U2 }: u$ VJan 14 03:30:34 localhost kernel: [ 10.661771] platform microcode: Falling back to user helper. ?! r8 H1 \9 y+ p+ v# ]6 ?
Jan 14 03:30:34 localhost kernel: [ 10.669655] platform microcode: Direct firmware load failed with error -2
3 z7 m' V+ l( Z! [Jan 14 03:30:34 localhost kernel: [ 10.669658] platform microcode: Falling back to user helper# H5 s5 [1 O5 U7 \+ o% T; T' w
Jan 14 03:30:34 localhost nrpe[2151]: Starting up daemon
6 z, B/ g2 n8 |+ N9 N9 G8 z6 }Jan 14 03:30:34 localhost nrpe[2151]: Network server bind failure (99: Cannot assign requested address)
* g2 c4 T8 z2 A% V* q3 jJan 14 03:30:34 localhost zebra[2157]: Zebra 0.99.20.1 starting: vty@0
9 c; n2 }7 Q5 Y% V6 TJan 14 03:30:34 localhost ripd[2168]: RIPd 0.99.20.1 starting: vty@0
9 Z6 [1 T2 H5 U ]) IJan 14 03:30:34 localhost ripngd[2170]: RIPNGd 0.99.20.1 starting: vty@0
( F0 @4 e' P4 j) p$ \1 _Jan 14 03:30:34 localhost ospfd[2172]: OSPFd 0.99.20.1 starting: vty@0
6 I: f; T% ~: o! DJan 14 03:30:34 localhost ospf6d[2174]: OSPF6d (Quagga-0.99.20.1 ospf6d-0.9.7r) starts: vty@09 ]! M5 j) P4 G* M+ }3 V2 b" a
Jan 14 03:30:34 localhost bgpd[2176]: BGPd 0.99.20.1 starting: vty@0, bgp@<all>:179
0 L/ x' u& D1 _6 [ Z8 _$ GJan 14 03:30:37 localhost zebra[2157]: interface eth0 index 2 changed <UP,BROADCAST,RUNNING,MULTICAST>.0 N5 y+ ?( S6 {- @. q& G) Q( g
Jan 14 03:30:38 localhost kernel: [ 17.283749] sshd (2567): /proc/2567/oom_adj is deprecated, please use /proc/2567/oom_score_adj instead.
: K2 ~* R0 Y/ n1 l
: K8 o& f$ |8 u. k5 r* Y; K% O: [* m5 W6 V* X
! r. ~$ Z) \# j& ]* q" ?现象：
, c) m, @1 w0 N三个措施。
: C j C, y- b/ n& Y: `- o1、增大内存
# J* v, A8 x8 S% {从12GB增大到16GB。+ M( h( N8 u2 p+ U# r1 Q* A
virsh setmaxmem vgfsxxx 16GB --config
9 Y" z. ]8 K5 `- e8 c/ e; I* A运行启动后
) X+ Z2 `2 e& h! M8 s: y* l2 Fvirsh setmem vgfsxxx 16GB
3 ^7 q2 i9 j1 v+ `5 Z这个没有用，跑了几次测试后，仍然掉服务。 2、调整lwfsd的服务优先级1 ]* ?, Y% _1 K" G% t ]
设置lwfsd的服务优先级为“-17”+ n6 o, [6 r- T7 g
PID=`ps | grep lwfs | grep -v grep | awk '{print $1}'`
, E D C8 y% j% J# t; becho -17 > /proc/$PID/oom_adj
( }* c1 J2 z; \% a2 N' xecho -17 > /proc/$PID/task/$PID/oom_adj
/ c% p% u' p4 K1 }这个好像有用。 3、修改内存分配策略8 O m6 _4 Z1 P+ K# G8 a! Z
并且echo "2" >/proc/sys/vm/overcommit_memory，使得分配内存时，必须存在足够的空间用于映射。# b$ e) G5 ]8 q/ j- p1 U
这个好像也有一定的用处。再跑跑试试。
, [) {1 r x3 n ?( W( i0 D$ F |
发表于 2020-1-15 09:00:03