|
|
关于OpenStack中虚拟机VNC访问安全问题
( S* ]) ~' X6 O4 @) `8 l/ K前几天收到甲方给出的文件信息,线上OpenStack环境的物理机VNC端口能够自由访问,要求整改。
7 n. w6 b2 D3 X: G随机抽查了两台宿主机的VNC端口,确实很多业务的同学使用noVNC后没有退出终端的习惯,往往都是用完了就直接关闭窗口。不得不说这样隐患很大啊,首先不说通过外部方式规避风险,如果内网里面有一些script kiddie随时都能将我们线上的虚拟机VNC端口扫出来干些坏事。我这里也用过nmap测试了下开发环境的网络端口,如下:% p! d& r& t7 z; h. f1 p
[root@controller1 ~]# nmap 10.161.53.1
) m3 P+ S d: S5 K' }Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
0 d! ?! C' N* G1 f, C- m6 W4 i$ ~Nmap scan report for compute10 (10.161.53.1)
" x8 }; t0 c" Z* e% LHost is up (0.000080s latency).
& i" `" L, F# }$ b! h% T& g/ t4 jNot shown: 989 closed ports
+ J1 e# Y K9 W; w9 B RPORT STATE SERVICE
+ ^1 Y* O% }7 b3 c2 M8 ~2 x22/tcp open ssh
C* D. I& O9 V+ q+ N7 }5900/tcp open vnc5 t$ v; F- V5 Z0 y" f( j4 T
5901/tcp open vnc-1$ z5 L/ l# w! T$ y
5902/tcp open vnc-2
) @, T; ~/ T0 {( k5903/tcp open vnc-3) T# i" X% v5 h& W5 a5 i# w3 ]
5904/tcp open unknown1 h, C; f) R! k ]$ [ g6 Q
5906/tcp open unknown
2 d. e$ u C0 L4 g* T9 N1 ^9 @ P' Y5907/tcp open unknown
) C3 F3 G0 S, B: @5910/tcp open cm
! z/ g& ~( f/ u0 D$ V' r5911/tcp open cpdlc
2 U X6 A3 U/ |: P1 n; K" U; Q8022/tcp open oa-system" u' B+ F4 E7 V/ r; A
MAC Address: D4:5D:64:08:45:02 (Unknown)
* e* H. {$ C2 @0 b& M" DNmap done: 1 IP address (1 host up) scanned in 1.65 seconds7 r" h- I6 y$ H+ F* G/ c- i: F& i
[root@controller1 ~]# nmap 10.161.53.2 B6 o* O* ]+ [' Q+ ~; l5 N$ o
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
( Q0 ^/ Y' h6 ]8 iNmap scan report for compute11 (10.161.53.2)4 M2 [3 h6 r' J6 y) e& i! `
Host is up (0.000070s latency).
$ C3 R- m( ?1 z) LNot shown: 989 closed ports
+ z( S8 D9 z2 _1 O- u# |PORT STATE SERVICE0 j L: S$ L4 ]5 T3 F. \
22/tcp open ssh
4 I# X& v$ N! b8 Y/ E5900/tcp open vnc
+ g8 `3 h3 J. E& T# p& M5901/tcp open vnc-1
y( y- M* D2 S9 y9 y5902/tcp open vnc-2: }# H3 j% o6 |! L, ` W
5903/tcp open vnc-3
7 _# S; \% M1 O5904/tcp open unknown
) u2 V; Z; _5 t4 x7 Z6 `5906/tcp open unknown
, B! p# V% e4 H( n; ]6 F) p5907/tcp open unknown1 b- ^+ ~/ a- y$ i. x/ I
5911/tcp open cpdlc- s( B/ h, `- ~' J8 {5 E6 M1 ~
5915/tcp open unknown( U" `% }" L/ U/ o; x) B/ B) e
8022/tcp open oa-system% [$ A, Y$ Y S4 y! Q
MAC Address: D4:5D:64:07:B3:DA (Unknown)
' ]& S( ~ N% DNmap done: 1 IP address (1 host up) scanned in 1.67 seconds
* ^# g' J* D, }$ k. b0 W[root@controller1 ~]# nmap 10.161.53.3
$ E! |4 s4 ]8 p+ Q. s. OStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST0 h$ f1 R6 u. X4 X0 l# x6 m
Nmap scan report for compute01 (10.161.53.3)) \8 Z4 H. L" Y4 w# S6 b' E
Host is up (0.000076s latency).
3 H2 ]8 O3 P$ j0 n9 p, b! s( U4 BNot shown: 992 closed ports: Y& B2 ~, j9 b0 ] n! \9 f
PORT STATE SERVICE3 f! k7 P5 C7 Y- r, i
22/tcp open ssh
) O; D. t/ ~2 {; {5 \5900/tcp open vnc. m, F3 ~' Y) [/ `0 Z; w$ d- z
5901/tcp open vnc-1- w* @ o& C+ R
5902/tcp open vnc-2. }; L7 F0 W% ?" w# p: D& `+ Y8 _0 M
5903/tcp open vnc-3$ E' y; s/ g; ?
5904/tcp open unknown
' O0 B+ i8 t: J; f2 {! O5906/tcp open unknown) V3 }$ S# O# p* c1 z9 |# o
8022/tcp open oa-system: t# o% [1 c& j" |. M
MAC Address: D4:5D:64:08:45:5E (Unknown)
& e4 w9 y7 ^1 Q T1 @Nmap done: 1 IP address (1 host up) scanned in 1.68 seconds
; I6 u: e7 `# B/ ~1 b3 z[root@controller1 ~]# nmap 10.161.53.4
. n# u7 D4 B- nStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST% I$ c$ g5 o3 U
Nmap scan report for compute02 (10.161.53.4)
) w" [3 O: p6 @* Q( ^Host is up (0.000081s latency)., w, e K" A" ]+ l' r7 Z
Not shown: 988 closed ports
# \+ Z# i" d& |/ EPORT STATE SERVICE
1 i# n! f7 q0 H+ E22/tcp open ssh& K7 Q* b; ]3 V9 s7 k# z$ J
5900/tcp open vnc
7 i: r$ I0 f' t6 c- X0 X5901/tcp open vnc-1
6 T O+ ~5 j) u, O$ C! U0 Z5902/tcp open vnc-2" S* n% L3 F, J
5903/tcp open vnc-31 L6 x u) C! C, ?
5904/tcp open unknown
* M( z6 @' O* x) J5906/tcp open unknown. P, l: U7 ?2 D& {6 x3 p* h$ C, S
5907/tcp open unknown
' x: G* z! t7 P" Y5910/tcp open cm/ \! o* b7 X& P! Q8 R7 L* e" _1 @
5911/tcp open cpdlc
& _: t0 O0 r" J5915/tcp open unknown+ \. C/ ~! g7 V4 A: y5 ? O
8022/tcp open oa-system! s1 E. i& b8 \: ^2 ~
MAC Address: D4:5D:64:08:45:D6 (Unknown)6 R3 R8 V9 X/ J& ~3 z
Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds
2 L' h) h5 J/ c4 d7 I$ p[root@controller1 ~]# nmap 10.161.53.5
9 V' e; N! T/ y( E5 x8 sStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST& q% I/ `8 q7 A
Nmap scan report for compute03 (10.161.53.5)
3 J* K5 N0 v7 Q- ?0 kHost is up (0.000082s latency).
! C1 O/ ]! ?+ W0 l$ f8 r" Z4 [; r8 |/ rNot shown: 993 closed ports
& X O: \, h5 h5 U' e/ I4 r! G$ bPORT STATE SERVICE
1 t$ E7 R1 T1 A. [! R( w, I# U22/tcp open ssh; [* t9 K7 p- ]3 F& P, a( d" }
5900/tcp open vnc
! z0 O K, ~3 J7 \# _5901/tcp open vnc-1: U/ \6 t; C% t7 I4 ~
5902/tcp open vnc-2
. S' y: i9 p, P4 s( \8 s( x. B5903/tcp open vnc-3
7 {1 H" q! B! @* E5904/tcp open unknown
! K/ V9 x0 ^0 j) W+ B8022/tcp open oa-system
! R& s, [- x0 \6 ^" [/ a$ NMAC Address: D4:5D:64:08:44:DE (Unknown) I8 h/ B# D4 L
Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds
' T+ j& v0 i0 g8 _6 V0 V' o0 l[root@controller1 ~]# nmap 10.161.53.60 ~3 O. a# U0 j2 `2 K7 N x8 m, G( I
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST' F$ z X1 |4 F$ ~- S
Nmap scan report for controller1 (10.161.53.6)! L D8 G5 V0 m4 V
Host is up (0.000014s latency).
3 l7 {" W- r' P" X& P1 aNot shown: 992 closed ports
3 E4 g& r2 f+ h+ X0 _# O, N' TPORT STATE SERVICE
' |8 J9 N- A; v% k& l22/tcp open ssh
. C4 N6 Q( W( O6 b# f! Q' W3 v" y8 n80/tcp open http
9 [0 p0 K R r7 K. c1984/tcp open bigbrother: x$ @8 k+ c# y& R' F! x Q
3306/tcp open mysql
5 n$ b" t9 Z8 f# |, T4000/tcp filtered remoteanything* F# v" x3 G- l& ~$ r I
4567/tcp open tram
k: y9 x; L2 l0 q% Z g5000/tcp open upnp) N, w. X2 g% R1 l
5001/tcp open commplex-link
3 x& V9 i9 K/ n& sNmap done: 1 IP address (1 host up) scanned in 2.75 seconds' U# Z1 g$ l' c; z) g0 s( d- x, o
[root@controller1 ~]# nmap 10.161.53.7# p) u7 o6 \3 q# ]" u! Y H
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST7 l' D4 C& @& W4 X+ Z' B" E& x2 d, D
Nmap scan report for controller2 (10.161.53.7)
6 T9 w/ r8 Q% D; P2 M, mHost is up (0.000078s latency).5 F8 o$ F: Z% w* C
Not shown: 993 closed ports
6 a! g* @7 o) PPORT STATE SERVICE
( T! Z; S; \ {- o1 W- d22/tcp open ssh
$ Q) W6 x( E" `' e) J80/tcp open http
3 c! u0 ]/ i7 l# Z1984/tcp open bigbrother) e9 S- G2 x4 X3 g7 h1 l {
3306/tcp open mysql
9 X5 g/ v* T ]$ o4 y- A/ P4567/tcp open tram1 {6 g- V K# Q+ u5 j+ y( W7 N
5000/tcp open upnp4 z# y9 _& u0 x9 L# M
5001/tcp open commplex-link$ h8 n4 [5 V) R; f. w
MAC Address: D4:5D:64:08:45:0A (Unknown)
; q. s$ R) P. X: K6 |0 qNmap done: 1 IP address (1 host up) scanned in 1.55 seconds; |5 y+ v& g5 u! {9 g( R4 {: y
) L6 \% |$ `, Q3 b
结果太恐怖了吧,如果有业务同学在使用noVNC之后没有退出终端,那么另一个人如果知道了宿主机的IP和端口是完全可以登录这台虚拟机的,直接操作虚机,后果不堪设想:
- C% X2 I" U: C2 E+ z! g; k9 B
$ j" ]" L; E0 K0 L! C7 F" Z k( \2 a解决5 P( m! q/ i E! j- c8 [5 R7 d
知道了问题,那就有对应的解决方案!& B# w4 ^ d6 n: P# z
目前我们暂时只想到两个方法来解决这个问题,其他方法还有待大牛给予指导:; f+ t. |- W2 l* i6 O' U! \4 V
) M% I6 B+ ] t/ o7 R p方案一
1 y# O5 e+ x. ` i通过firewalld限制INPUT表对5900:6000的访问规则$ p+ G: Z: | n: g; f
方案二* x5 [9 V7 Q& q, j% ~- T3 w' p
添加密码访问VNC
* o3 x7 Q" `. R% I, L' u2 Y/ v& m" |7 N' j% |' a' Y
操作
) z# X- Y: ^5 \- f* ^firewalld- D$ p7 f- H* u% R
我们知道OpenStack通过VNC Proxy将管理网和业务网隔离开来,以便我们可以使用管理网络的6080端口访问虚拟机VNC,同时提供Token用于验证访问的合法性。一个VNC Proxy在OpenStack里的处理流程如下:; X" @9 e+ M3 m$ B3 ] G6 w
VNC Porxy处理流程
* U( x9 _2 G3 t1. 一个用户试图从浏览器里面打开连接到虚拟机的VNC Client
) ?" C7 i, |3 p/ P! e2. 浏览器向nova-api发送请求,要求返回访问vnc的url
' |2 R. d; w. n. H9 i3. nova-api调用nova-compute的get vnc console方法,要求返回连接VNC的信息2 N" ~1 v6 c$ i: q
4.nova-compute调用libvirt的get vnc console函数
+ s) L0 W" B6 Z6 K+ C- }5.libvirt会通过解析虚拟机运行的/etc/libvirt/qemu/instance-0000000c.xml文件来获得VNC Server的信息
3 k- l$ \/ l9 h! N" N1 X6.libvirt将host, port等信息以json格式返回给nova-compute- Y( ~9 s# b5 a5 _
7.nova-compute会随机生成一个UUID作为Token) D; I% v0 y) ^% J; J& x
8.nova-compute将libvirt返回的信息以及配置文件中的信息综合成connect_info返回给nova-api
, r' G( O }$ G$ \7 O- E3 O ]9.nova-api会调用nova-consoleauth的authorize_console函数
5 R4 L5 g* T. {) {1 H _10.nova-consoleauth会将instance –> token, token –> connect_info的信息cache起来
/ S5 [& P- p. @3 f+ s11.nova-api将connect_info中的access url信息返回给浏览器:http://contorller:6080/vnc_auto.html?token=7efaee3f-eada-4731-a87c-e173cbd25e98&title=helloworld%289169fdb2-5b74-46b1-9803-60d2926bd97c%29
3 a% `+ e5 b1 }4 l& a12.浏览器会试图打开这个链接: ]- [ C- \ }5 @" d0 w' N8 V2 B
13.这个链接会将请求发送给nova-novncproxy
) ?2 l( {* f& T. t5 T% g- e I- q( ]14.nova-novncproxy调用nova-consoleauth的check_token函数% T* Q4 e( Z$ G' x" W
15.nova-consoleauth验证了这个token,将这个instance对应的connect_info返回给nova-novncproxy
) s, S! `; u. F2 U: _16.nova-novncproxy通过connect_info中的host, port等信息,连接compute节点上的VNC Server,从而开始了proxy的工作5 ]1 f9 n, e( g7 s* W9 M0 m
这里重要的就是第16步, nova-novncproxy是通过连接host:vncport的方式提供vnc访问服务。
( \, T: I% P& ?7 n3 m; U+ C- b那么也就是说,计算节点的VNC端口只需要让nova-novncporxy服务能够访问就行,有了这个就好办了。
/ C: k$ Z6 [& J' _! ?& E( Y操作firewalld |) z. W, [) Y. f) M9 W1 c% o' E
在所有计算节点firewalld的INPUT表中添加如下规则:
7 E( G3 L( M- k. @6 z! v4 ~) h; y* r8 F. `: k& v' v1 Q
检查防火墙是否启动:
8 ?% D; s% X8 M* ?systemctl status firewalld.service
# p+ f9 _9 H0 \! N+ A启动防火墙:% ~! t2 T8 I$ \* ^" @
systemctl start firewalld.service
- v& z% @; O' g! V. E
9 o3 z% l( [/ u8 B) u# Y+ v
$ a* S* e! g: u. ~, N' O* a6 w开机启动防火墙:% |3 s& y8 Y$ `( i: ^, ]* u
systemctl enable firewalld.service) i' p2 ^, n- F$ J4 `/ _% `
添加规则:4 s* l: K" d, J. |
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept"
7 l/ m- ^; V4 f! A8 P+ w6 U/ L7 R firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.100" port protocol="tcp" port="5900-6000" accept"
! |0 J4 b2 \, B. L( Y: ^! I firewall-cmd --reload
& A& C9 m4 n6 }. d' r i3 J firewall-cmd --list-all
- b9 i f3 _* t, E; ]删除规则:
$ i3 s% P: ^) Q/ v1 N5 T firewall-cmd --permanent --remove-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept"5 K+ I9 ^4 `7 e) e- g: ?2 o
6 j9 \5 {% y3 m" O: Z# y, C
firewall-cmd --reload 6 Z5 o7 a) p, S
firewall-cmd --list-all5 D5 j& D( c D: E. X
9 m% Y: V/ @ V& [
. ?9 p4 A! l8 S- M3 L+ L9 D! [( j$ Z添加防火墙规则:
8 O' \- u. J$ @) b9 yfirewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.31/27" port protocol="tcp" port="5900-6000" accept"
* h- T1 {; M0 e; K+ ]. s9 E1 J, ^* i( B5 U# u8 r
firewall-cmd --reload ' k9 ^! k+ k" v$ Z" M0 z
% Z2 T7 g z- k# f1 Q通过测试上面开启的firewalld会导致业务无法访问。 C4 S; d' @' p1 V% ~ ^
这里改变方式使用iptables的规则吧:. U \) s3 V. y! W4 q4 s9 r
" u' J# a; S0 [( X" T9 e; K) S# W
iptables -A INPUT -s 10.161.53.31/27 -p tcp -m multiport --dports 5900:5999 -m comment --comment "ACCEPT VNC Port only by Controller Node" -j ACCEPT
/ a3 o& O- D) {" T g iptables -L
# X5 O# i! @% z7 ? iptables -A INPUT -p tcp -m multiport --dports 5900:5999 -j REJECT --reject-with icmp-port-unreachable
5 k, B5 G# |" p1 S6 o9 [4 @# B
* w( L5 d# v" z
" H8 z% D% V5 ?1 Y( o+ D: P) @# p( d) ?' z& t/ K8 M5 H
|
|