t1 ] _& T- V; r O& e+ sJan 14 04:06:25 localhost kernel: [9821120.248545] ------------[ cut here ]------------
3 ?1 S5 E! w5 n7 ~8 r' TJan 14 04:06:25 localhost kernel: [9821120.248573] WARNING: CPU: 2 PID: 13942 at fs/dcache.c:362 d_shrink_del+0x2c/0x6e()
2 y. B. Y4 a; ^0 t8 DJan 14 04:06:25 localhost kernel: [9821120.248584] Modules linked in: deflate ctr twofish_generic twofish_x86_64_3way twofish_x86_64 twofish_common camellia_generic serpent_sse2_x86_64 ablk_helper cryptd glue_helper lrw serpent_generic xts gf128mul blowfish_ge9 ?. ?5 M j- a. v$ x% {* @# H# A
neric blowfish_x86_64 blowfish_common cast5_generic cast_common des_generic cbc xcbc rmd160 sha512_generic sha256_generic crypto_null xfrm_user xfrm4_tunnel tunnel4 ipcomp xfrm_ipcomp esp4 ah4 af_key xfrm_algo 8021q mrp garp stp llc cls_u32 sch_htb xt_nat xt_LOG xt_comm$ x! T2 ` e7 c9 U* a0 E6 g' r' e
ent nf_conntrack_netlink iptable_nat nf_nat_ipv4 ip6table_filter ip6table_raw ip6_tables iptable_filter nf_conntrack_ipv4 nf_defrag_ipv4 xt_CT nfnetlink_cthelper nfnetlink iptable_raw nf_nat_pptp nf_conntrack_pptp nf_conntrack_proto_gre nf_nat_h323 nf_conntrack_h323 nf_% a' Z, \% R/ M& D1 g6 n
nat_sip nf_conntrack_sip nf_nat_proto_gre nf_nat_tftp nf_nat_ftp nf_nat nf_conntrack_tftp nf_conntrack_ftp nf_conntrack cpufreq_userspace cpufreq_stats cpufreq_powersave cpufreq_ondemand cpufreq_conservative ipv6 fuse i2c_piix4 i2& {+ a& E0 {/ P, Q7 Q! \4 K1 h5 F [
Jan 14 04:06:25 localhost kernel: [9821120.249030] CPU: 2 PID: 13942 Comm: sh Not tainted 3.13.11-1-amd64-vyos #1
0 f3 o1 _& v$ o9 u( ]$ ]Jan 14 04:06:25 localhost kernel: [9821120.249043] Hardware name: Alcatel-Lucent ACCS NFV r1.0 HA, BIOS 0.5.1 01/01/2011) U8 K8 l: M8 n0 i
Jan 14 04:06:25 localhost kernel: [9821120.249054] 0000000000000000 ffffffff814f58a3 ffffffff813eb0b1 0000000000000000* p* y" I& X7 T. N5 i H8 S: p
Jan 14 04:06:25 localhost kernel: [9821120.249125] ffffffff8104a614 ffff880215609380 ffffffff81133665 ffffffff813edfd2
# u. H* N+ d: @$ ]" y- ~Jan 14 04:06:25 localhost kernel: [9821120.249143] ffff880080cc66c0 ffff880080cc6718 ffff880231b1dce0 ffff880231b1dca0: H% ]6 F O5 H2 c, [! a f9 V8 @) H
Jan 14 04:06:25 localhost kernel: [9821120.249161] Call Trace:/ z/ [0 v" n% I
Jan 14 04:06:25 localhost kernel: [9821120.249183] [<ffffffff813eb0b1>] ? dump_stack+0x41/0x58
, u$ I# U8 j. c# DJan 14 04:06:25 localhost kernel: [9821120.249208] [<ffffffff8104a614>] ? warn_slowpath_common+0x79/0x927 |7 h6 w0 h6 }1 f5 T
Jan 14 04:06:25 localhost kernel: [9821120.249230] [<ffffffff81133665>] ? d_shrink_del+0x2c/0x6e
4 H( H+ J8 p9 B+ k, [3 L/ o, O t. zJan 14 04:06:25 localhost kernel: [9821120.249250] [<ffffffff813edfd2>] ? _raw_spin_unlock+0x5/0x6
1 ~% d" E$ u8 O- IJan 14 04:06:25 localhost kernel: [9821120.249273] [<ffffffff81133665>] ? d_shrink_del+0x2c/0x6e
, ~/ @" n r. i8 ^9 }/ BJan 14 04:06:25 localhost kernel: [9821120.249295] [<ffffffff81134667>] ? shrink_dentry_list+0x51/0xc6
. m3 R: H+ o- |9 v% ]1 ZJan 14 04:06:25 localhost kernel: [9821120.249315] [<ffffffff813edfd2>] ? _raw_spin_unlock+0x5/0x6
5 a; \1 Q1 J1 C3 g7 P) TJan 14 04:06:25 localhost kernel: [9821120.249339] [<ffffffff811347a5>] ? shrinJan 14 03:30:34 localhost kernel: [ 0.000000] Linux version 3.13.11-1-amd64-vyos (jenkins@squeeze64devel) (gcc version 4.4.5 (Debian 4.4.5-8) ) #1 SMP Wed Aug 12 02:08:05 UTC 2015: C ^8 z( X. O% v- B0 ^" H
Jan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: RSDP 00000000000f70d0 000014 (v00 BOCHS )
; I# i: O8 D8 X+ u) Z2 q& yJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: RSDT 00000000bffff70a 00002C (v01 BOCHS BXPCRSDT 00000001 BXPC 00000001)* [/ U6 N8 ]2 G, f6 ?) `
Jan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: FACP 00000000bffff606 000074 (v01 BOCHS BXPCFACP 00000001 BXPC 00000001)" M2 x' d/ \ j0 q* k3 K/ b' x3 T
Jan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: DSDT 00000000bfffe040 0015C6 (v01 BOCHS BXPCDSDT 00000001 BXPC 00000001)
. `% E: F! f1 V; Q$ p5 G$ G# f( YJan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: FACS 00000000bfffe000 000040 d5 U- }0 u/ M9 n/ U1 I
Jan 14 03:30:34 localhost kernel: [ 0.000000] ACPI: APIC 00000000bffff67a 000090 (v01 BOCHS BXPCAPIC 00000001 BXPC 00000001)) |7 x, E- z, E3 v X! {
Jan 14 03:30:34 localhost kernel: [ 0.000000] Zone ranges:, j0 Y8 {2 H( t0 V
Jan 14 03:30:34 localhost kernel: [ 0.000000] DMA [mem 0x00001000-0x00ffffff]
+ |$ ?7 F7 ^5 r0 ?/ H3 p2 n0 j* MJan 14 03:30:34 localhost kernel: [ 0.000000] DMA32 [mem 0x01000000-0xffffffff]
9 M3 K0 \9 |1 z$ I# ~' w5 `: \Jan 14 03:30:34 localhost kernel: [ 0.000000] Normal [mem 0x100000000-0x23fffffff]: [. H. E, @- j- h: d P
Jan 14 03:30:34 localhost kernel: [ 0.000000] Movable zone start for each node4 n- i- i0 l8 N6 u6 g3 ?
Jan 14 03:30:34 localhost kernel: [ 0.000000] Early memory node ranges0 u3 _( F; ~1 g7 ]1 s; x! C
Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x00001000-0x0009efff]
9 i/ B) f- w5 f3 f: m7 b* [Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x00100000-0xbfff9fff]4 B) v* h/ ?" @2 @9 {5 [. _! u8 {
Jan 14 03:30:34 localhost kernel: [ 0.000000] node 0: [mem 0x100000000-0x23fffffff]
: S# Q* q8 w3 \. k/ qJan 14 03:30:34 localhost kernel: [ 0.000000] Built 1 zonelists in Zone order, mobility grouping on. Total pages: 2064258" G _) C7 D. L3 h8 Q7 s `
Jan 14 03:30:34 localhost kernel: [ 0.000000] Policy zone: Normal
/ M2 s3 v3 h; e# H1 j ^+ `% s2 ~Jan 14 03:30:34 localhost kernel: [ 0.000000] Kernel command line: BOOT_IMAGE=/boot/1.1.7/vmlinuz boot=live quiet vyatta-union=/boot/1.1.7 console=ttyS0,9600 console=tty0
0 B3 m- Z) y" f" s9 G; IJan 14 03:30:34 localhost kernel: [ 0.000000] Memory: 8167636K/8388192K available (4043K kernel code, 594K rwdata, 1652K rodata, 976K init, 700K bss, 220556K reserved). P$ ~0 T5 B4 ]! ^9 V& D
Jan 14 03:30:34 localhost kernel: [ 0.040964] ACPI: All ACPI Tables successfully acquired
5 N8 T D) z# x0 @! J! uJan 14 03:30:34 localhost kernel: [ 0.050000] Failed to access perfctr msr (MSR c1 is 0)
( A9 {$ h/ F9 ^4 [+ T1 s3 C7 EJan 14 03:30:34 localhost kernel: [ 0.142534] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S1_] (20131115/hwxface-580)) t( l/ t$ Z. o) L
Jan 14 03:30:34 localhost kernel: [ 0.142541] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S2_] (20131115/hwxface-580)7 O& {. W! _1 C1 s
Jan 14 03:30:34 localhost kernel: [ 0.142545] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S3_] (20131115/hwxface-580)
5 j0 J5 n( ~" }+ uJan 14 03:30:34 localhost kernel: [ 0.142549] ACPI Exception: AE_NOT_FOUND, While evaluating Sleep State [\_S4_] (20131115/hwxface-580): _% H0 Y/ ?2 h6 [- x
Jan 14 03:30:34 localhost kernel: [ 0.150892] acpi PNP0A03:00: fail to add MMCONFIG information, can't access extended PCI configuration space under this bridge.
* t8 _: @8 I, vJan 14 03:30:34 localhost kernel: [ 0.443990] ACPI: Enabled 3 GPEs in block 00 to 0F
/ q4 N) x4 g3 |3 z* X( L6 AJan 14 03:30:34 localhost kernel: [ 0.444104] SCSI subsystem initialized
* h: X# T2 E1 p% F) C& Y- X: JJan 14 03:30:34 localhost kernel: [ 0.532368] ACPI: PCI Interrupt Link [LNKD] enabled at IRQ 11
% q0 G) A1 q) }) w. p# e( k9 {: pJan 14 03:30:34 localhost kernel: [ 1.028215] type=2000 audit(1578972622.313:1): initialized
$ l4 r# T. r) Z: C' P# W: m" _8 [Jan 14 03:30:34 localhost kernel: [ 1.029615] bounce pool size: 64 pages
2 v. U' g4 x8 s' s- M2 t4 F! ~Jan 14 03:30:34 localhost kernel: [ 1.138358] Key type dns_resolver registered/ O7 A" D6 P5 F, l- _$ }1 F
Jan 14 03:30:34 localhost kernel: [ 1.244013] ACPI: PCI Interrupt Link [LNKC] enabled at IRQ 10
% l0 U3 l" ]9 L8 p1 j) vJan 14 03:30:34 localhost kernel: [ 1.357808] ACPI: PCI Interrupt Link [LNKA] enabled at IRQ 10
/ O5 u/ O- q9 y6 [9 q, }; {3 R6 V% vJan 14 03:30:34 localhost kernel: [ 2.040051] raid6: sse2x1 4639 MB/s
" D' ^( O$ X" |" a% `Jan 14 03:30:34 localhost kernel: [ 2.210027] raid6: sse2x2 6362 MB/s
, M7 r4 p2 w( F! ~Jan 14 03:30:34 localhost kernel: [ 2.380032] raid6: sse2x4 7357 MB/s
8 ?" c/ V6 ?: h6 W8 G4 dJan 14 03:30:34 localhost kernel: [ 2.380037] raid6: using algorithm sse2x4 (7357 MB/s)
# `& {' Q0 ?* ZJan 14 03:30:34 localhost kernel: [ 2.380040] raid6: using intx1 recovery algorithm( y) z2 }, p. m; ^" W5 j6 O
Jan 14 03:30:34 localhost kernel: [ 8.178340] EXT4-fs (vda1): warning: checktime reached, running e2fsck is recommended
4 k/ N, g( P! L8 L, e2 wJan 14 03:30:34 localhost kernel: [ 9.087041] random: debconf-communi urandom read with 25 bits of entropy available
4 T: A" }3 m) q( }Jan 14 03:30:34 localhost kernel: [ 10.625650] platform microcode: Direct firmware load failed with error -29 |+ k a1 U! C
Jan 14 03:30:34 localhost kernel: [ 10.625654] platform microcode: Falling back to user helper! y: t! n) g, ?/ u' K" L! Y6 H4 o
Jan 14 03:30:34 localhost kernel: [ 10.652304] platform microcode: Direct firmware load failed with error -2
) o8 v5 I6 S9 N" N1 ]5 x7 WJan 14 03:30:34 localhost kernel: [ 10.652309] platform microcode: Falling back to user helper
# ~1 w7 T( ?' o6 g+ ~2 d$ h* x, JJan 14 03:30:34 localhost kernel: [ 10.661767] platform microcode: Direct firmware load failed with error -2
' b( G- x- Z ? P& SJan 14 03:30:34 localhost kernel: [ 10.661771] platform microcode: Falling back to user helper
, ^ D d4 i. s2 ?7 MJan 14 03:30:34 localhost kernel: [ 10.669655] platform microcode: Direct firmware load failed with error -2
/ k" B: }4 E$ R& b$ D/ V- I, } w& ?7 L6 fJan 14 03:30:34 localhost kernel: [ 10.669658] platform microcode: Falling back to user helper& ]) ]8 F a, O" L
Jan 14 03:30:34 localhost nrpe[2151]: Starting up daemon! w7 X8 Q. ^+ T) G6 v
Jan 14 03:30:34 localhost nrpe[2151]: Network server bind failure (99: Cannot assign requested address)
8 f% E) R! J* Z: k! wJan 14 03:30:34 localhost zebra[2157]: Zebra 0.99.20.1 starting: vty@0
0 m* o- H6 F$ s' Y* v! f/ mJan 14 03:30:34 localhost ripd[2168]: RIPd 0.99.20.1 starting: vty@0
; u4 P. ?: s$ v7 ^* k2 j; JJan 14 03:30:34 localhost ripngd[2170]: RIPNGd 0.99.20.1 starting: vty@0
" o3 c8 K. a9 g/ iJan 14 03:30:34 localhost ospfd[2172]: OSPFd 0.99.20.1 starting: vty@06 c6 m$ [9 K+ b u3 t! D
Jan 14 03:30:34 localhost ospf6d[2174]: OSPF6d (Quagga-0.99.20.1 ospf6d-0.9.7r) starts: vty@0
% i2 f U7 q, @' PJan 14 03:30:34 localhost bgpd[2176]: BGPd 0.99.20.1 starting: vty@0, bgp@<all>:179
, \. t: x0 D: y; j! W' _Jan 14 03:30:37 localhost zebra[2157]: interface eth0 index 2 changed <UP,BROADCAST,RUNNING,MULTICAST>., N) d+ J0 Y# R( u' T
Jan 14 03:30:38 localhost kernel: [ 17.283749] sshd (2567): /proc/2567/oom_adj is deprecated, please use /proc/2567/oom_score_adj instead./ O4 S& X; F' q9 h- [- n' u+ h
" V, @2 C" {8 Z# \. |$ y5 H( @
' `& B* V; ?+ P' e6 V, r9 f" K) p2 L
9 m" P1 m) u1 ?0 N `; T& L现象：6 K; }% A7 F3 P2 L" b9 t8 U
三个措施。
& _2 t2 q4 y* B: b4 K) H1、增大内存
2 Z. d x+ @4 G$ u% }; q从12GB增大到16GB。- O1 O$ a) ]3 a) E4 f- b3 ?2 s
virsh setmaxmem vgfsxxx 16GB --config6 J! g5 A2 l, }/ r# f
运行启动后
! ?' W3 v0 a+ y- y& E; mvirsh setmem vgfsxxx 16GB( {! k: C* ^5 {' S; h) N
这个没有用，跑了几次测试后，仍然掉服务。 2、调整lwfsd的服务优先级
' d! i4 f8 h$ I1 H设置lwfsd的服务优先级为“-17” f' h- o8 J9 T1 E5 z8 R
PID=`ps | grep lwfs | grep -v grep | awk '{print $1}'`- U+ G. u# y" o) l
echo -17 > /proc/$PID/oom_adj" {! j; P* z: b# \+ _% ]1 \
echo -17 > /proc/$PID/task/$PID/oom_adj9 ^4 t8 d0 Q* @7 B
这个好像有用。 3、修改内存分配策略
. I9 ?3 L2 G, z% @# n5 z7 d5 |并且echo "2" >/proc/sys/vm/overcommit_memory，使得分配内存时，必须存在足够的空间用于映射。
; i8 {3 N7 O/ g! p; I# F* ^/ d这个好像也有一定的用处。再跑跑试试。 ! V: x8 E2 O! {# r: H* q/ ~
|
发表于 2020-1-15 09:00:03