openstack平台dashboard出现闪退

admin

dashboard的闪退问题
[Sun Sep 02 13:31:09.172447 2018] [:error] [pid 28024] INFO openstack_auth.plugin.base Attempted scope to domain default failed, will attemptto scope to another domain.
, a) K9 _. r# r# ~) m2 t[Sun Sep 02 13:31:09.346659 2018] [:error] [pid 28024] INFO openstack_auth.forms Login successful for user "admin" using domain "default", remote address 192.168.90.1.

  [! A2 O- T+ i+ m% _: I# X% ~[Sun Sep 02 13:34:25.577781 2018] [authz_core:error] [pid 28419] [client 192.168.90.1:57704] AH01630: client denied by server configuration: /usr/bin/keystone-wsgi-public, referer: http://192.168.90.65/dashboard/auth/login/
" A3 q+ Y( V6 |  B5 v. c
openstack-status
8 B; S+ @4 q" F& [3 O! L# ?== Nova services ==
2 C& |' q$ X) b: v" R2 q& [: Copenstack-nova-api:                     active
openstack-nova-compute:                 inactive  (disabled on boot)
openstack-nova-network:                 inactive  (disabled on boot)
7 ?+ c% P+ z& V2 m  e. Z+ lopenstack-nova-scheduler:               active
. Z5 K( N( d6 Z4 C) w  K3 nopenstack-nova-conductor:               active
openstack-nova-console:                 inactive  (disabled on boot)
4 |* C& U  R$ F3 \. D& popenstack-nova-consoleauth:             active
0 U" i8 h$ m' r5 Gopenstack-nova-xvpvncproxy:             inactive  (disabled on boot)
2 x& L7 y7 l1 Z+ T0 j5 W== Glance services ==
' _; ^: {( x3 B6 L" U! iopenstack-glance-api:                   active
5 y5 g  J  R- i2 oopenstack-glance-registry:              active
== Keystone service ==
openstack-keystone:                     inactive  (disabled on boot)
5 F- S/ U6 a2 _3 j% y== Horizon service ==
openstack-dashboard:                    uncontactable
== neutron services ==
neutron-server:                         active
5 S: ]9 d4 C$ ~( H/ Fneutron-dhcp-agent:                     active
& M. }( C9 _* J: |) kneutron-l3-agent:                       active
neutron-metadata-agent:                 active
( F/ N/ z  f) Q4 y1 g8 z% ?6 J: [neutron-linuxbridge-agent:              active
1 F0 s7 Y  }  D3 F0 [+ |! k1 z== Support services ==
mariadb:                                active
dbus:                                   active
4 h/ k1 x1 |' c( l2 i$ u- {rabbitmq-server:                        active
memcached:                              active
/ ^( K* z6 w$ y- l== Keystone users ==
+----------------------------------+-----------+
0 S5 w: I( B9 k- d% K7 E| ID                               | Name      |
+----------------------------------+-----------+
| 35cba790e2b44595a9219e6c18176065 | nova      |
5 ~! E  U' f0 _/ f2 F6 i4 m  n| 72177ec2346542518bb5e5a14f2745d4 | glance    |
| 893fd35e580c4d54a8c775d84207b191 | admin     |
| 91506d00e6e14cec99bf448cbb189a9a | neutron   |
| a47170e629444ae59e655f11f92022d9 | placement |
! J+ t/ k7 E8 }| af71b1d71cc54262aee370e9b6d5c622 | demo      |
. x* l$ ]/ \  h: F+----------------------------------+-----------+
: ^2 g+ ^) O: N% \( |) P== Glance images ==
) q: d5 `4 F4 j5 q+ |' Y3 [+--------------------------------------+---------------------+
5 Z1 N0 y1 Z7 G| ID                                   | Name                |
0 G$ _% ?( j9 f. G4 a+--------------------------------------+---------------------+
| 047884ba-6a47-45c0-8ace-d72c5d566367 | cirros              |
| 2c685860-4fa2-43dd-abad-1cae69e59861 | cirros-0.3.4-x86_64 |
8 D, J1 G  n3 f* G% D) C+--------------------------------------+---------------------+
== Nova managed services ==
+--------------------------------------+------------------+------------+----------+---------+-------+----------------------------+-----------------+-------------+
| Id                                   | Binary           | Host       | Zone     | Status  | State | Updated_at                 | Disabled Reason | Forced down |
" e5 M3 F% v0 }8 o: ^8 W" ~7 D+--------------------------------------+------------------+------------+----------+---------+-------+----------------------------+-----------------+-------------+
| 11290f6e-283f-4309-884e-7a171ab0a53e | nova-consoleauth | controller | internal | enabled | up    | 2018-09-02T05:35:44.000000 | -               | False       |
; Z# I0 J/ C1 V, M| 1c161768-160b-4829-9245-1ba3598ab5a4 | nova-scheduler   | controller | internal | enabled | up    | 2018-09-02T05:35:41.000000 | -               | False       |
( o1 ~+ a0 T1 D& O5 u5 \9 l4 X| 6d66755b-76bf-481d-a4ec-1ccddd8641c5 | nova-conductor   | controller | internal | enabled | up    | 2018-09-02T05:35:47.000000 | -               | False       |
| 8d12536b-22d2-44e7-a1d5-656d20ff0393 | nova-compute     | compute    | nova     | enabled | up    | 2018-09-02T05:35:49.000000 | -               | False       |
+ I* P; D' U8 \; M1 |7 ~) m: W+--------------------------------------+------------------+------------+----------+---------+-------+----------------------------+-----------------+-------------+
== Nova networks ==
0 h+ b% V& y: b3 x/ d6 Ousage: nova [--version] [--debug] [--os-cache] [--timings]
2 C9 ^1 I+ j/ D6 ]; Y) |7 n; U            [--os-region-name <region-name>] [--service-type <service-type>]
+ X! a8 d; T  C/ x! w. w/ C            [--service-name <service-name>]
" G, x6 i# K) l% p) A! g! f            [--os-endpoint-type <endpoint-type>]
            [--os-compute-api-version <compute-api-ver>]
. I+ R  {' y+ N$ C            [--endpoint-override <bypass-url>] [--profile HMAC_KEY]
            [--insecure] [--os-cacert <ca-certificate>]
8 h4 B0 e7 A# f+ Z: K1 [  `            [--os-cert <certificate>] [--os-key <key>] [--timeout <seconds>]
! W3 t7 Q# A/ M7 c& ]            [--os-auth-type <name>] [--os-auth-url OS_AUTH_URL]
$ D, j1 o/ {7 P# y. z& o            [--os-system-scope OS_SYSTEM_SCOPE] [--os-domain-id OS_DOMAIN_ID]
            [--os-domain-name OS_DOMAIN_NAME] [--os-project-id OS_PROJECT_ID]
7 D' @8 D3 m( g6 q1 _  D            [--os-project-name OS_PROJECT_NAME]
            [--os-project-domain-id OS_PROJECT_DOMAIN_ID]
            [--os-project-domain-name OS_PROJECT_DOMAIN_NAME]
9 s+ }% i8 E+ a! E! b            [--os-trust-id OS_TRUST_ID]
            [--os-default-domain-id OS_DEFAULT_DOMAIN_ID]
            [--os-default-domain-name OS_DEFAULT_DOMAIN_NAME]
            [--os-user-id OS_USER_ID] [--os-username OS_USERNAME]
1 l/ M. ]/ V& c0 R2 _+ ^            [--os-user-domain-id OS_USER_DOMAIN_ID]
            [--os-user-domain-name OS_USER_DOMAIN_NAME]
            [--os-password OS_PASSWORD]
            <subcommand> ...
error: argument <subcommand>: invalid choice: u'network-list'
Try 'nova help ' for more information.
/ Y% v8 j( \- X1 d% N* {== Nova instance flavors ==
& S( R$ U$ }+ [' ?. x+----+------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
| ID | Name | Memory_MB | Disk | Ephemeral | Swap | VCPUs | RXTX_Factor | Is_Public | Description |
* ^  V* T% k( e3 p+----+------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
+----+------+-----------+------+-----------+------+-------+-------------+-----------+-------------+
: C/ C' j* {8 e== Nova instances ==
5 Q) H( K. K5 J+----+------+-----------+--------+------------+-------------+----------+
| ID | Name | Tenant ID | Status | Task State | Power State | Networks |
4 j' }6 T9 W4 [+----+------+-----------+--------+------------+-------------+----------+
+----+------+-----------+--------+------------+-------------+----------+
8 j, `3 F3 v8 f7 ~

admin

### 重启httpd memcached rabbitmq-server 和neutron服务，还是 无法修复

1 i- v4 f  a, S/ U- l" j/ ?查看keystone

openstack service list
' v  h- q( A9 t+ P4 i: {+----------------------------------+-----------+-----------+
| ID                               | Name      | Type      |
/ y7 f& y: \4 X6 m! P2 j# `* ]+----------------------------------+-----------+-----------+
( [/ v) q/ n* G7 `% ]| 02929a487f484e9fbb2221d11212b11f | neutron   | network   |
4 z  [6 H' R  p: r5 {0 G6 A| 1b0061ff680042ea8bd764de0cefc25a | placement | placement |
5 E* e( m3 s% W2 n- {& u| 48cc7da12bb84f118ed528c57ba1cb3c | nova      | compute   |
| 75581fbfdf244aa78b5aba2a656a53e5 | keystone  | identity  |
| ae94823f314c46098f1ff04dba0427db | glance    | image     |
# U9 P4 [/ O  q* _  I6 i+----------------------------------+-----------+-----------+
  b: R, ]# y! o3 t1 y/ g' p; I" T

没有发现重复的demain
) [3 i) s4 e& Y- T
############重启控制节点，   节点无法修复
$ B4 \  r6 x1 b, k

• systemctl restart rabbitmq-server.service
  ( l6 _7 x% q& Q+ v9 H6 t
  - k! @/ b& q6 X0 r  h
• 
  - C% Y3 _. e2 c5 t* C: @7 z( B
  systemctl restart neutron-server.service                 
  
  ; B6 T& y6 ^+ s! j4 U/ J1 D
• 
  4 j7 b  M: o. k* l2 }
  systemctl daemon-reload
  
  & f) Y4 x" d7 X) c
• 
  
  systemctl stop httpd.service memcached
  / g8 x/ y: }- B3 N8 B
  4 R% c& Z% n- g+ O! y9 A5 y
• 
  
  5 }" M, @; u8 A$ C最后搞定
  
  ! S" x) t  ]# K6 C4 F
• 
    }2 ^/ z/ |) s
  

$ f/ y5 G( {" t% }0 ^; l' c8 y

心得：由于本环境是多控节点，rabbitmq-server 集群 没有优化，出现故障较多，建议如果发现有amqp 相关报错，先检查 rabbitmq 服务，有问题就重启一下

admin

=ERROR REPORT==== 2-Sep-2018::13:43:24 ===
closing AMQP connection <0.1199.0> (192.168.90.65:48940 -> 192.168.90.65:5672):
{handshake_error,starting,1,
                 {amqp_error,unexpected_frame,
7 R* ]. ]4 [$ t, P                             "type 1, first 16 octets = <<0,60,0,40,0,0,38,114,101,112,108,121,95,57,48,50>>",
9 {% D/ o, C; ?  |8 Q7 g                             none}}

admin

keystone  access信息：
( Z3 u. D1 J* q' A192.168.90.65 - - [02/Sep/2018:13:47:26 +0800] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 (internal dummy connection)"
192.168.90.65 - - [02/Sep/2018:13:47:34 +0800] "OPTIONS * HTTP/1.0" 200 - "-" "Apache/2.4.6 (CentOS) mod_wsgi/3.4 Python/2.7.5 (internal dummy connection)"
- Y3 w; S! g! K3 \* h# J* {2 z
也没有发现问题，尴尬

admin

后来测试发现，使用Firefox浏览器好像可以减少dashboard页面闪退现象。这是浏览器的问题么？