|
|
关于OpenStack中虚拟机VNC访问安全问题
" K$ b1 D9 c+ f* \4 i% [前几天收到甲方给出的文件信息,线上OpenStack环境的物理机VNC端口能够自由访问,要求整改。" z+ i# T" w2 l# U) l# v
随机抽查了两台宿主机的VNC端口,确实很多业务的同学使用noVNC后没有退出终端的习惯,往往都是用完了就直接关闭窗口。不得不说这样隐患很大啊,首先不说通过外部方式规避风险,如果内网里面有一些script kiddie随时都能将我们线上的虚拟机VNC端口扫出来干些坏事。我这里也用过nmap测试了下开发环境的网络端口,如下:
3 Y) Y) J. Z& K @) c* J' M[root@controller1 ~]# nmap 10.161.53.1% T! b Q" E( L! o) u
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
+ `6 N) G) m" T' U2 ]Nmap scan report for compute10 (10.161.53.1), W* X7 D+ G1 L9 B/ h% N( K5 w
Host is up (0.000080s latency)., S* t5 [. `* h
Not shown: 989 closed ports8 H! K a; D3 X S
PORT STATE SERVICE+ q1 F& x$ r4 E
22/tcp open ssh
( {, _) N' E. {( u1 F/ B5900/tcp open vnc
$ ^9 t/ Z7 B" K, k2 r$ z3 p5901/tcp open vnc-1
* A* U" @. Y. d) U( ?8 O5902/tcp open vnc-27 C: x$ I3 N% j9 V6 ^; D
5903/tcp open vnc-3
& ]- r2 ^" e4 c9 p" G5904/tcp open unknown
" V8 y' _/ S d. J8 O! ^5906/tcp open unknown
8 {# Y* a. M2 W4 |' S5907/tcp open unknown
% J; h* k; e7 t: y6 |5910/tcp open cm7 q6 [' s/ |1 @( l9 k; u4 K
5911/tcp open cpdlc
3 [+ h2 a9 m8 O# S8022/tcp open oa-system( D' g- M; l1 ?, ^
MAC Address: D4:5D:64:08:45:02 (Unknown)
& U& M& X$ x* C0 ]% U& MNmap done: 1 IP address (1 host up) scanned in 1.65 seconds& k* E3 a5 v1 X, M! |9 C: w3 b
[root@controller1 ~]# nmap 10.161.53.2
1 E- [5 v# \4 c! xStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST0 d& T' Z; ~! h0 s; f
Nmap scan report for compute11 (10.161.53.2)9 {6 P8 X4 h- N. W+ b
Host is up (0.000070s latency).* I4 J; Z0 v$ \# q+ Q' n
Not shown: 989 closed ports
8 j2 Z4 X! y7 B- m, OPORT STATE SERVICE. {! u! r% D' z/ Z3 H3 o+ k D
22/tcp open ssh
6 R$ W2 X) S" n3 m5900/tcp open vnc
" x1 [4 U* _ b, ^5 {5901/tcp open vnc-1* M9 A2 m7 ? P# {3 m
5902/tcp open vnc-2
0 l9 `' h4 x7 h$ v T9 {5903/tcp open vnc-3
7 u0 E& k" K( ^2 l. o- n4 j3 P5904/tcp open unknown
# h! B t7 z T, j1 \) ]: l! v4 _2 K5906/tcp open unknown7 ?, T% _ C+ D" R D
5907/tcp open unknown
: r- W5 B$ s: z: `3 S! P5911/tcp open cpdlc
" I9 b0 W4 M% t: ~% L5915/tcp open unknown
8 q: N# [& @. q" q8022/tcp open oa-system
" O% w! m, |/ w, B) AMAC Address: D4:5D:64:07:B3:DA (Unknown)" A4 @2 B$ p* C5 Q7 ?6 s& T2 u
Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds
& U4 y& w% d% R% V S# T8 c) A% d[root@controller1 ~]# nmap 10.161.53.3) k# r3 j" z) U" X' H+ g+ v2 a
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST
6 r( f+ ]: D/ I% ?) FNmap scan report for compute01 (10.161.53.3)& g) E7 W& j( y* I7 W, m3 S
Host is up (0.000076s latency)., k8 g3 [! J* @1 q
Not shown: 992 closed ports1 l! a" o6 {6 C/ q; B5 V! l
PORT STATE SERVICE
* n0 L: A$ ]1 F' I$ B* E22/tcp open ssh# ]# t/ A8 v% Q( R/ c
5900/tcp open vnc; g q7 e2 J% q/ D
5901/tcp open vnc-1
( A- h: N5 N, x" r8 P5902/tcp open vnc-2/ @6 Y0 G$ `: x) ~& k8 q8 K
5903/tcp open vnc-3# r+ {% V- ~1 T$ W- @
5904/tcp open unknown% Y) Z, J# f" o2 W& l" I t' L6 k
5906/tcp open unknown G4 F$ i( x- `- ~( s/ A: f
8022/tcp open oa-system
3 f" ] ?5 }" K: Z9 tMAC Address: D4:5D:64:08:45:5E (Unknown)# x7 k) a5 V) f% \
Nmap done: 1 IP address (1 host up) scanned in 1.68 seconds, H" z( O4 l" d$ j
[root@controller1 ~]# nmap 10.161.53.4% A" g; R8 q, m) E5 j# B" C
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:46 CST- Z; ]8 |- w1 E
Nmap scan report for compute02 (10.161.53.4)
* Q% i" G# Z, I; t' P# z" q( }- FHost is up (0.000081s latency).
2 g0 i$ s8 V/ t/ m) Y$ K! T0 SNot shown: 988 closed ports
. ^3 M0 F( K9 @3 H1 v! a. tPORT STATE SERVICE
, c& i( r* b( v% H" Y22/tcp open ssh( K L: ]1 ?. ?+ i/ v) S- c. H
5900/tcp open vnc
7 X7 r4 O( g. [* B# {5901/tcp open vnc-1( n3 d4 n- p, Y; J5 U# L, G
5902/tcp open vnc-2
4 v b M9 F/ f' E i3 |5903/tcp open vnc-3
+ m* H( I7 V, Z* O: t: {7 y( n5904/tcp open unknown* A: ^$ }: M' \' j; V. P
5906/tcp open unknown. K$ g4 e/ j0 z. ^& C
5907/tcp open unknown& }: B" m! z6 R1 [& {
5910/tcp open cm
- A# }% Q [+ a! Z6 N8 q5911/tcp open cpdlc7 f2 k6 {' ~. o+ S; [
5915/tcp open unknown1 t8 j" w" F# V; ~# r3 m
8022/tcp open oa-system
2 k D+ I q1 n3 q" B: T8 w5 [2 YMAC Address: D4:5D:64:08:45:D6 (Unknown)
5 z0 [& Q {6 n" P5 B; SNmap done: 1 IP address (1 host up) scanned in 1.67 seconds* w7 b; X8 e+ J6 D
[root@controller1 ~]# nmap 10.161.53.5# T$ T' A1 X; D2 O
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST
8 j" h& @% v2 j- X6 x& U' c2 `( RNmap scan report for compute03 (10.161.53.5)5 W2 R, t! X: j4 j% S8 G0 ?% A( A
Host is up (0.000082s latency).+ k( v4 ^5 X# Z/ V
Not shown: 993 closed ports
& `7 c5 t% h0 o" }3 i" p; v% fPORT STATE SERVICE
- r- u/ d( h& ]! d22/tcp open ssh
0 s2 @) }* a/ ?5900/tcp open vnc, j3 D8 h5 D6 M' K2 m
5901/tcp open vnc-1
2 h" Z3 ^+ R# i0 G( D) B# M* x5902/tcp open vnc-2
( |; j6 P0 {& s% L5 s5903/tcp open vnc-31 L0 a, Q% Q/ R
5904/tcp open unknown
5 W* J6 |! n: V: k5 q3 \8022/tcp open oa-system- e% ]/ c$ O8 T4 ]0 N! p) o
MAC Address: D4:5D:64:08:44:DE (Unknown)
' ]# t7 p8 O7 @Nmap done: 1 IP address (1 host up) scanned in 1.67 seconds9 s" P& N% o8 E
[root@controller1 ~]# nmap 10.161.53.6
+ n( C, F+ O* Y5 y# ^# C0 w- MStarting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST/ S+ z6 ?2 r: Y( X( A! S
Nmap scan report for controller1 (10.161.53.6)
1 y! ~: V, \7 b8 ]3 t, U; iHost is up (0.000014s latency).
# T' W' ]( a9 _! w# \Not shown: 992 closed ports
6 ^; s8 G( `# W7 D2 dPORT STATE SERVICE
5 Y# L6 c. p# y( \7 f22/tcp open ssh
8 x# K N$ P6 _80/tcp open http
6 n A; ~* F0 [) s( G1984/tcp open bigbrother$ r/ S) J3 n1 X' g
3306/tcp open mysql+ L# j2 j9 t- O; V
4000/tcp filtered remoteanything
) O: `! b- u- v" C" J w4567/tcp open tram& @; I+ Z% {8 l$ I& ]- R
5000/tcp open upnp
. [/ a% i, W1 q; W4 q5001/tcp open commplex-link
/ Z8 z% O, u( Z! {, P4 u6 d' wNmap done: 1 IP address (1 host up) scanned in 2.75 seconds
\1 p# a% N$ k" ?/ v2 }2 |# d[root@controller1 ~]# nmap 10.161.53.76 ]; h& _! P. y5 O$ D
Starting Nmap 6.40 ( http://nmap.org ) at 2022-01-17 09:47 CST+ C/ h5 m" C' N& L$ R: j% j. ]
Nmap scan report for controller2 (10.161.53.7)
+ _+ R- ?7 }, d D$ W8 gHost is up (0.000078s latency).& U3 x, w4 E6 k( ]
Not shown: 993 closed ports
8 G3 Z( z2 _! ~: Y2 d2 Q% aPORT STATE SERVICE5 W; T2 Y7 a' o& t' X0 i8 X, `
22/tcp open ssh( e0 e2 Y2 B0 F
80/tcp open http
) |) q# _$ V2 U1984/tcp open bigbrother
$ m% X: { [) K3306/tcp open mysql
9 f8 A( Y' o8 z! O3 _4567/tcp open tram& I. R+ c" Z, E$ d! E
5000/tcp open upnp- M" d! d% ]+ |
5001/tcp open commplex-link
/ E# z: D) X3 I% l4 H+ z VMAC Address: D4:5D:64:08:45:0A (Unknown)
7 S6 Z' `' A5 _7 e& vNmap done: 1 IP address (1 host up) scanned in 1.55 seconds& b- ~% T2 [, u T& ]% o" h
0 ?" _+ f/ P2 Z; t结果太恐怖了吧,如果有业务同学在使用noVNC之后没有退出终端,那么另一个人如果知道了宿主机的IP和端口是完全可以登录这台虚拟机的,直接操作虚机,后果不堪设想:
* p7 o8 \3 [& `! K' \
) ~( T$ `; ]7 r4 X解决' D& C. M B/ E# ^2 H6 d
知道了问题,那就有对应的解决方案!
' U* _3 M5 G$ R# e+ N- |目前我们暂时只想到两个方法来解决这个问题,其他方法还有待大牛给予指导:
" A! P) R# ~6 D Z; P9 |$ S( O3 W. j9 c
方案一" Y4 [- C6 K! P1 [" d. q( A
通过firewalld限制INPUT表对5900:6000的访问规则
^# N# B! r! C8 L. D方案二3 k) T {' { F% @, u" e
添加密码访问VNC) R8 {, V9 b2 C# }3 r1 k; e; b
5 C" [' W. y S/ k操作
5 K; h0 e+ ?( z; {* ]firewalld% Y. d# k, J( B
我们知道OpenStack通过VNC Proxy将管理网和业务网隔离开来,以便我们可以使用管理网络的6080端口访问虚拟机VNC,同时提供Token用于验证访问的合法性。一个VNC Proxy在OpenStack里的处理流程如下:
* K) f: a, p0 y2 AVNC Porxy处理流程$ @3 \" ]. L5 s+ _$ _; n3 _6 Y8 ?, I1 Q
1. 一个用户试图从浏览器里面打开连接到虚拟机的VNC Client* [$ v) d/ A/ M' y C! y ~
2. 浏览器向nova-api发送请求,要求返回访问vnc的url
9 r T7 _- p+ P/ l& {/ [3. nova-api调用nova-compute的get vnc console方法,要求返回连接VNC的信息- h/ W. A/ B/ l( ^
4.nova-compute调用libvirt的get vnc console函数4 p% @" g4 D7 v; v/ T
5.libvirt会通过解析虚拟机运行的/etc/libvirt/qemu/instance-0000000c.xml文件来获得VNC Server的信息3 m, d# e9 P5 p5 q& U2 x& d
6.libvirt将host, port等信息以json格式返回给nova-compute1 Y7 J# M: t- ?! i1 V# ^
7.nova-compute会随机生成一个UUID作为Token9 H1 \, q6 U% r6 W8 ^% d& H
8.nova-compute将libvirt返回的信息以及配置文件中的信息综合成connect_info返回给nova-api
$ W/ d" [: c- ]- C- e; c+ ?" [3 x5 A9.nova-api会调用nova-consoleauth的authorize_console函数
5 A' {$ j2 x( \2 U! e/ b; r10.nova-consoleauth会将instance –> token, token –> connect_info的信息cache起来
* u0 _' I4 T# y6 U! ^/ v' a3 H11.nova-api将connect_info中的access url信息返回给浏览器:http://contorller:6080/vnc_auto.html?token=7efaee3f-eada-4731-a87c-e173cbd25e98&title=helloworld%289169fdb2-5b74-46b1-9803-60d2926bd97c%29# p2 i& P* M2 r; \
12.浏览器会试图打开这个链接
$ z' c% E) E L5 ~% A, S8 m13.这个链接会将请求发送给nova-novncproxy* n' K* N% q P) W1 `7 V2 V( ?- P
14.nova-novncproxy调用nova-consoleauth的check_token函数
- Y8 J7 D! H2 h: ]* N5 j15.nova-consoleauth验证了这个token,将这个instance对应的connect_info返回给nova-novncproxy: T! h" |' K0 j# i. s9 X7 O8 K
16.nova-novncproxy通过connect_info中的host, port等信息,连接compute节点上的VNC Server,从而开始了proxy的工作2 i$ @# X7 X7 O7 m- r
这里重要的就是第16步, nova-novncproxy是通过连接host:vncport的方式提供vnc访问服务。
. A- I/ [+ }, ^& w那么也就是说,计算节点的VNC端口只需要让nova-novncporxy服务能够访问就行,有了这个就好办了。4 I* w6 G4 S) S! P# G3 G, @. r
操作firewalld
$ f7 z0 t$ h$ J1 b8 ]1 R) H# t在所有计算节点firewalld的INPUT表中添加如下规则:
( |5 h$ Z1 w, Q- r5 |5 e1 j; Z# \9 J% W: @6 `& ^- y8 l _6 h
检查防火墙是否启动:
5 t# i4 M: \0 y6 A( r; s1 xsystemctl status firewalld.service
7 D, U; @5 ~% c# n/ {启动防火墙:
7 U6 z: N! v( Y6 g. fsystemctl start firewalld.service
3 \6 [) V7 n0 z# {+ g# V! I+ O/ y3 k l( B
2 c1 X+ H; R- t4 V3 c) u7 r
开机启动防火墙:. `' z. @3 N% i; c+ r3 G2 }
systemctl enable firewalld.service
( i8 w4 d) p3 r+ n! k: R4 |添加规则:
8 a( o: w/ \" Q& P1 S4 z+ C) j firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept"; _* F1 ?4 H) P% Y4 n# }
firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.100" port protocol="tcp" port="5900-6000" accept"" f- v d" q8 c6 T4 G
firewall-cmd --reload - a6 G0 ?! n8 C* \% a
firewall-cmd --list-all
& U5 @! O9 n& w: `删除规则:
" t* R& ]$ a. K# o( e0 ^5 P+ b2 F firewall-cmd --permanent --remove-rich-rule="rule family="ipv4" source address="10.161.53.6" port protocol="tcp" port="5900-6000" accept": V9 j1 a5 ]! M) z) l! q
' q, Z+ @$ \# [/ N. U1 [* ]
firewall-cmd --reload ! j: M9 N3 t* r+ S1 |* \
firewall-cmd --list-all( K+ o8 {; q3 V3 q- V
7 F8 n* e% B$ X( x
3 n; m9 A; X( R' W# v" n- [; X
添加防火墙规则:
! Y$ e- |3 g6 G$ @7 T |firewall-cmd --permanent --add-rich-rule="rule family="ipv4" source address="10.161.53.31/27" port protocol="tcp" port="5900-6000" accept"5 ^# T# ]8 w% Z2 Q; A" W
9 _0 q. ]$ n0 O" P& \8 g: u
firewall-cmd --reload 3 L+ g5 v$ s- J& x! x
4 w! D/ m6 r6 I( {- |% \
通过测试上面开启的firewalld会导致业务无法访问。* D: Y+ V- @4 z
这里改变方式使用iptables的规则吧:: f( J4 O: g3 Y6 ]* g/ S5 f6 H
. {" W) F' F0 ?8 m0 G1 E+ m
iptables -A INPUT -s 10.161.53.31/27 -p tcp -m multiport --dports 5900:5999 -m comment --comment "ACCEPT VNC Port only by Controller Node" -j ACCEPT
8 o$ D+ e" ^! i+ i ?7 ] iptables -L& b- Q9 n2 h; U; K6 @3 v
iptables -A INPUT -p tcp -m multiport --dports 5900:5999 -j REJECT --reject-with icmp-port-unreachable5 ~# x, ]- u% {2 @( E( ?
$ H7 L/ P# F/ q5 L& u( }
+ _" c, W. B9 R, A$ z. P' n2 S5 x# v7 ^
|
|