ceph luminous 提供iscsi服务

admin

开始安装之前必须要满足以下条件：
5 _; ]" ]* N# ^) x6 \. H7 s+ L正在运行的Ceph Luminous版本集群或更高版本的存储群集；（本文我们采用的是M版本）
RHEL/CentOS 7.5或者Linux kernel v4.16或更新；
同时拥有以下几个包：
9 B9 ?! ]$ k: m0 `2 t+ U' stargetcli-2.1.fb47 or      newer package
( z/ l7 w" n5 bpython-rtslib-2.1.fb64 or      newer package
tcmu-runner-1.3.0 or      newer package
ceph-iscsi-config-2.4 or      newer package
: F, `: C3 G7 ^) c/ r, t) \0 Y! ]ceph-iscsi-cli-2.5 or      newer package
+ x. M% B8 W# {! ^$ j  x; v8 Q注意：我们本次安装ISCSI GW是在OSD 节点上安装
6 d" N: k# [( g* J6 f由于很多童鞋可能获取不到rpm包，所以我这里整理了一下所需要的rpm并且分享了出来，以下是链接
5 K, b$ k" U, O链接: https://pan.baidu.com/s/1piNymp6oWLNHvXKaxBszLg 密码: 4vag
安装部分就不说了只要把rpm包下载到本地，安装很简单，下面来说下配置部分：
首先创建一个iscsi gw的配置文件
# touch /etc/ceph/iscsi-gateway.cfg
里面内容如下：
[config]
! p# g( z- O2 H4 G9 g# Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
# access to the Ceph storage cluster from the gateway node is required, if not
# colocated on an OSD node.
: x% y& S' A5 S; O4 h# icluster_name = ceph

# Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
6 @3 `- @2 E1 h5 S6 p- Z7 Y& L# drectory and reference the filename here
/ q. a& M2 A4 h, g% x( Ggateway_keyring = ceph.client.admin.keyring
- u2 o- D' p! ^, V
& n* g! @$ \4 k2 o3 S) a
( e/ [; }& c' L2 y# API settings.
4 b) l% X$ \+ r  N# The API supports a number of options that allow you to tailor it to your
+ r' x, T# V8 ^5 g3 R# local environment. If you want to run the API under https, you will need to
# create cert/key files that are compatible for each iSCSI gateway node, that is
  y& E( H% [' m* V# not locked to a specific node. SSL cert and key files *must* be called
# 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
- I# V4 ~4 A( k4 d' b. {; u# on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
7 i5 @$ h, T) u6 Q  H  p# to switch to https mode.

: C1 w( z7 a- I% \# To support the API, the bear minimum settings are:
api_secure = false

# Additional API configuration options are as follows, defaults shown.
& j, {$ q1 n2 M' m# api_user = admin
' E: I8 N1 _' T5 D% u4 Y# api_password = admin
$ T" {4 A8 S% y4 X  c# api_port = 5001
+ W3 d. E1 h" G# trusted_ip_list = 192.168.0.10,192.168.0.11
注意：需要修改的地方就行把trusted_ip_list替换成自己实际场景的节点ip
) u# K, n# V6 h完事之后重启iscsi服务
: Z1 _$ v8 v; x# systemctl daemon-reload
$ P" t4 X) i" o8 a: O. i# systemctl enable rbd-target-api
2 D" J: a1 P, i0 C( j5 X. H8 o4 Z# systemctl start rbd-target-api
% y7 L9 l- Y9 y+ }OK到此为止，iscsi gw已经安装配置完成，下面开始创建iscsi target
0 o+ a0 Y( N0 u. B# _- Q
1.执行gwcli命令进入iscsi

2.创建iscsi target
) _7 N: H7 A- A. r. ?> /> cd /iscsi-target
* u4 Q3 F0 k+ c+ B# R; t: u3 Q> /iscsi-target>  create iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw

3.创建iSCSI网关。以下使用的IP是用于iSCSI数据传输的IP,它们可以与trusted_ip_list中列出的用于管理操作的IP相同，也可以不同，看有没有做多网卡分离
> /iscsi-target> cd iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/gateways
8 M2 q9 Y. z& x- u3 B* G> /iscsi-target...-igw/gateways>  create ceph-gw-1 10.172.19.21 skipchecks=true
: Z! Z) _2 w: i6 m9 R% m> /iscsi-target...-igw/gateways>  create ceph-gw-2 10.172.19.22 skipchecks=true
+ l9 P# a* }* Q  A% n  n) I* `0 y8 _
4.创建一个rbd设备disk_1
+ @- |5 O1 Q0 t> /iscsi-target...-igw/gateways> cd /disks
0 Y* S! A' C: l  X/ Q3 b/ M> /disks> create pool=rbd image=disk_1 size=90G
; v2 Z7 `/ Z$ ?2 w5 S6 v
$ l& Q/ L) s  {3 @% ?# R4 ]; g5.创建iqn
: N# c5 ^9 e/ D5 `$ H: A; Y> /disks> cd /iscsi-target/iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/hosts
> /iscsi-target...eph-igw/hosts>  create iqn.1994-05.com.redhat:rh7-client

6.创建chap的用户名密码，并且chap必须设置，否则服务端是禁止连接的
) V+ t3 Q# `+ y% Q> /iscsi-target...at:rh7-client>  auth chap=myiscsiusername/myiscsipassword

1 K4 f- @) l# J9 h4 ^7.增加磁盘到客户端
> /iscsi-target...at:rh7-client> disk add rbd.disk_1
5 J: ~& A1 i  N2 S2 Y/ F4 k* t8 e
9 _$ ?: {$ j7 M  V$ ^2 w' [9 I% S, ]OK，到这里就配置完成了，我们看下最终是什么样的
) k7 c" i/ Z  @0 M: `. I/ F' Liscsi.png
# ]& E% c5 v& T; L* @" m) m! Q
安装配置完之后就可以挂载使用了，一般也就是windows/Linux，我们先说下Linux下挂载方式
1 z8 b' i5 h) A$ [+ R
0 l4 Z& v/ i" {8 Y# o8 P1.安装iscsi工具
9 A3 m5 I  f/ w' v7 F' O* N# yum install iscsi-initiator-utils
. m* U! p! M! t' o2 ^$ N# yum install device-mapper-multipath
- }* C$ ?+ n9 ]9 d% n3 X  K
! S4 h" S" F- u- T& n- Z2 P2.配置多路径
; \/ ]3 d* ?. n; e# mpathconf --enable --with_multipathd y
; x3 n- o2 c' K( i# l# d1 }devices {
        device {
6 X) ?& V4 E5 j% v) y                vendor                 "LIO-ORG"
                hardware_handler       "1 alua"
                path_grouping_policy   "failover"
; \: _( Q* W* }* b1 v  e                path_selector          "queue-length 0"
& i8 u0 i1 W8 u) A                failback               60
) e) L: |# P3 l( w, D                path_checker           tur
7 c0 X8 p4 M& ]' l; B+ n                prio                   alua
" f" P$ }8 f  O- D$ V. I# x                prio_args              exclusive_pref_bit
                fast_io_fail_tmo       25
% ^9 L$ i8 O4 S                no_path_retry          queue
' Y% _  d5 x9 K! i0 M        }
}
# O+ h6 r4 ]8 o& D; f5 ~
. U/ M# I$ M$ H0 B/ ^3.重启服务
# systemctl reload multipathd
" P' H2 d  `8 J& Y2 }3 o
4 c. d7 W7 O, s: ?& Q. w+ w0 O; ?4.配置chap认证
修改配置客户端的名称为之前设置的名称
# vi /etc/iscsi/initiatorname.iscsi
1 ~* a9 c) s% _8 R! Y& Z/ \) } InitiatorName= iqn.1994-05.com.redhat:rh7-client
" U- N/ d; J1 I1 l! k修改chap认证配置文件
, _2 f1 ^9 K. n: n # vi /etc/iscsi/iscsid.conf
7 X  \" Z! Q0 V8 P# n node.session.auth.authmethod = CHAP
node.session.auth.username = myiscsiusername
  C  H/ O( p6 ]. i/ `! M node.session.auth.password = myiscsipassword

5.查询iscsi target
# iscsiadm -m discovery -t -st 10.172.19.21
  B8 o1 _3 r" N% t
6.连接target
# iscsiadm -m node -T iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw -l
# multipath –ll
) }. I2 s0 b7 ~3 ?5 K
' G! {. p- W/ l) F8 g, Z. ]% q

admin

[root@ceph01 ~]# vi /etc/yum.repos.d/CentOS-Base.repo
) g6 p9 |! ^, y& o; l3 e
, z& e0 ?- W+ {[Storage]
$ c0 J: q4 }& P6 |name=CentOS-$releasever - Storage
baseurl=https://mirrors.huaweicloud.com/centos/7.7.1908/storage/x86_64/ceph-nautilus/
7 @* O3 e& v' ~& f9 ~( v0 S  L" C7 D: Ngpgcheck=0
5 Q7 T1 E  V7 P' J1 c! @enabled=1
, l3 v; K& q) a
[Gluster]
name=CentOS-$releasever - Storage
baseurl=https://mirrors.huaweicloud.com/centos/7.7.1908/storage/x86_64/gluster-5/
( A8 U5 y; B& b" c5 ]2 C  K) zgpgcheck=0
" q" L9 t" q3 I2 H* V! yenabled=1

[ceph-iscsi]
  s% K. L2 O7 {5 K7 [2 s: G# hname=CentOS-$releasever - Storage
0 e" B2 k- V( m- j- P1 A- e# sbaseurl=https://mirrors.huaweicloud.com/ceph/ceph-iscsi/3/rpm/el7/noarch/
gpgcheck=0
enabled=1

[epel]
6 B, L7 n" A8 T9 J" R: Zname=CentOS-$releasever - Storage
5 j. C) O3 Q. e% I1 m8 o4 U. rbaseurl=https://mirrors.huaweicloud.com/epel/7Server/x86_64/
gpgcheck=0
5 s7 e! P; L4 w  l; g- Aenabled=1
9 ~* c( |0 i: c

admin

直接yum install试试可以发现，只有targetcli和python-rtslib能装上，而且版本都比官网说的要低，好吧，麻烦来了。
4 C/ {4 J$ a0 x$ i
: m: e. ?% c8 z经过一段几个小时*的搜索……从下面链接中找到了rpm包：
% e( K3 ?- X- \
Build new RPM for 3.0
; `% Q7 k" Y, ^, R* X% u3 O" ?Missing ceph-iscsi-cli package
3 D5 h9 J( ^; N新建repo文件：

0 u  Q( _. d7 a# ]  m: M sudo vim /etc/yum.repo.d/iscsi.repo
cat /etc/yum.repo.d/iscsi.repo
/ `8 v) x4 {6 |7 F8 |' c2 ]
[ceph-iscsi]
name=Ceph-iscsi
; R0 t$ x+ n( i" M  I! Fbaseurl=https://4.chacra.ceph.com/r/ceph-iscsi/master/88f3f67981c7da15448f140f711a1a8413d450b0/centos/7/flavors/default/noarch/
3 i! R; R. U( dpriority=1
& `% j3 e( u+ z) i* N, P& K& P6 vgpgcheck=0

[tcmu-runner]
name=tcmu-runner
  C" N5 I  g% X" A% J  [9 l2 T% qbaseurl=https://3.chacra.ceph.com/r/tcmu-runner/master/eef511565078fb4e2ed52caaff16e6c7e75ed6c3/centos/7/flavors/default/x86_64/
priority=1
gpgcheck=0

7 U0 ^. j- _/ O9 G: R0 P- l$ s  [[python-rtslib]
7 T$ x7 k& x" G$ p  z+ X- Z  K" @name=python-rtslib
baseurl=https://2.chacra.ceph.com/r/python-rtslib/master/67eb1605c697b6307d8083b2962f5170db13d306/centos/7/flavors/default/noarch/
priority=1
5 l6 n$ g  x- I: J0 tgpgcheck=0
这里我使用的是本地源，将上面的包下载到本地源：
: F. h2 N- g( v4 o! |2 @
sudo yum install --downloadonly --downloaddir=yum/ceph-iscsi/ targetcli python-rtslib tcmu-runner ceph-iscsi
1 s: d) q+ C! Y. B9 U createrepo -p -d -o yum/ yum/
4 f7 Y# p# M& y$ ~9 B: J注意到这里没有包含targetcli的repo，因为没有找到，使用yum基础的Base源或者是Ceph源可以安装targetcli-2.1.fb46-7.el7.noarch.rpm， 虽然官网需要的是targetcli-2.1.fb47 or newer package，但在后续使用中发现没有影响，所以这里就不用管targetcli了。
7 T. ~& o& A  B, a% Q" x; n7 H
9 e- S% @' a' c/ R那么这里下载下来的就是：

targetcli-2.1.fb46-7.el7.noarch.rpm
python-rtslib-2.1.fb68-1.noarch.rpm
5 Q- J+ ]; V5 }tcmu-runner-1.4.0-0.1.51.geef5115.el7.x86_64.rpm
1 G) {  q, y& n7 mceph-iscsi-3.2-8.g88f3f67.el7.noarch.rpm
ceph-iscsi网关初始配置
9 C9 y2 j5 m5 k& W如果使用的不是集群内的节点作为ceph-iscsi网关，那就需要进行一些初始的配置。
; }( |. D) ]" E% {+ Q1 X
安装ceph。
从集群中的一台机器上拷贝/etc/ceph/ceph.conf到本机的/etc/ceph/ceph.conf。
从集群中的一台机器上拷贝/etc/ceph/ceph.client.admin.keyring到本机的/etc/ceph/ceph.client.admin.keyring。
9 h0 C8 O! Y6 A' Q" W2 l当然第2和3步可以直接在deploy节点使用ceph-deploy admin {node-gateway}，{node-gateway}就表示网关节点的名字。

1 M+ H. d8 Z5 ?& M0 f( k: W$ \% |. M2 t可见这里就是为了将ceph-iscsi网关节点变成一个admin节点。

/ Q' a4 r6 q( |5 Z% D, s这时在网关节点上应该可以执行相关命令操作ceph集群，例如sudo ceph -s查询当前集群的状态。
+ {  y( \! a6 j
安装配置iscsi
这里官网建议先切换到root用户，方便一点：

0 p/ V) d+ D1 R9 n' h  d su root
在两个网关节点上都安装iscsi（注意到上面已经将相关包下载到了本地源，所以可以直接yum安装）：
3 j$ Z# E. ?' s' {, Y
) N  |; I( b+ T# c' Z' U7 C yum install -y ceph-iscsi
服务启动：

' w. ~) R# r! r. Z先创建rbd pool，如果没有的话。

ceph osd lspools
- {: }+ T/ Q- g$ h
: \; m0 s" A/ H2 T1 cephfs_data
- r( p+ G/ f  }3 ?- O' D: F2 cephfs_metadata

5 i4 B) M9 J; I' u ceph osd pool create rbd 128

pool 'rbd' created
; o) q- i. H  ~3 P- k3 ]# V创建并修改/etc/ceph/iscsi-gateway.cfg文件：
5 J5 y5 R3 Z2 t9 ?& \
vim /etc/ceph/iscsi-gateway.cfg
% C+ E4 _- N1 o. L( p$ T: K cat /etc/ceph/iscsi-gateway.cfg

[config]
: C: F: ]( e  |# j+ x7 S' N Name of the Ceph storage cluster. A suitable Ceph configuration file allowing
* R  q0 w: C! z+ W# Z; |6 T8 l access to the Ceph storage cluster from the gateway node is required, if not
colocated on an OSD node.
. Y* P' x) A4 l/ hcluster_name = ceph
5 c, Q- ]  }0 w
4 x+ t1 ~( f% h3 h$ h7 e+ O Place a copy of the ceph cluster's admin keyring in the gateway's /etc/ceph
4 ~' |, z" x: Y  j drectory and reference the filename here
gateway_keyring = ceph.client.admin.keyring


/ G* g. H" B2 Q& ]4 l) h/ E* | API settings.
The API supports a number of options that allow you to tailor it to your
& d1 n4 U2 Z+ d2 j! M! R) |$ ? local environment. If you want to run the API under https, you will need to
create cert/key files that are compatible for each iSCSI gateway node, that is
- E( N* Y1 n! p+ ~+ \ not locked to a specific node. SSL cert and key files *must* be called
" |, R9 u, C5 I% P1 B; S* ]/ e 'iscsi-gateway.crt' and 'iscsi-gateway.key' and placed in the '/etc/ceph/' directory
on *each* gateway node. With the SSL files in place, you can use 'api_secure = true'
to switch to https mode.
/ j7 _. B1 V7 K" f
8 Y* E7 [: n5 a6 e" V0 [ To support the API, the bear minimum settings are:
api_secure = false
9 @/ [/ I# k6 d- E7 F( p
3 ~2 h& s; S6 M1 O Additional API configuration options are as follows, defaults shown.
api_user = admin
7 s" d4 y& Y! v: r+ h+ T( A/ C api_password = admin
9 i- F9 P! |9 P& T: D( J& g9 o api_port = 5001
trusted_ip_list = 192.168.90.234,192.168.90.235
上面trusted_ip_list填写的就是两台网关的ip（这里不讨论多网卡的情况）。

+ o# J4 B, a* l, G8 a& d  H; A在另外一台网关上复制这个文件：

sudo scp cluster@node2:/etc/ceph/iscsi-gateway.cfg /etc/ceph/iscsi-gateway.cfg
在两台网关上都开启rbd-target-api服务：

systemctl daemon-reload
systemctl enable rbd-target-api
systemctl start rbd-target-api
配置：（在其中一台网关配置就行）
, @0 @1 m( f5 s
. w& A; ^0 ^4 F& Q gwcli
cd /iscsi-targets

create iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw
% D, p9 I* p4 l; o% y! a, k4 X/ T" Rok

cd iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/gateways
% }9 z6 U6 u2 W
/iscsi-target...-igw/gateways> create node2 192.168.90.234
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
9 B0 e- W% `4 Wok

/iscsi-target...-igw/gateways> create node3 192.168.90.235
Adding gateway, sync'ing 0 disk(s) and 0 client(s)
ok

/iscsi-target...-igw/gateways> cd /disks
9 X/ F# F) C/ s/ X1 p
. F3 r1 |2 _) o! ]4 b0 c create pool=rbd image=disk_1 size=200G
0 P# t$ u1 N$ r, T$ T' eok

1 g' Y3 o9 a: ^" G' [- \ cd /iscsi-targets/iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw/hosts

, \4 k: e  o: ^0 C+ q: \# z* `/iscsi-target...csi-igw/hosts> create iqn.1994-05.com.redhat:rh7-client
ok
; M) W9 A8 _, o8 ^5 e. }
/iscsi-target...at:rh7-client> auth username=myiscsiusername password=myiscsipassword
ok
( p; |4 K" p: s0 E. d
) o1 Y( J+ J& z6 O- O/iscsi-target...at:rh7-client> disk add rbd/disk_1
, A  M' z' `" i8 u8 |' P( v8 yok
配置完成，可以看到我当前的目录结构：
; n6 m2 V, g. `# U- e$ @! ^2 V
ls
o- / ......................................................................... [...]
  o- cluster ......................................................... [Clusters: 1]
/ u3 n3 b! Y; c, L& Y3 {  | o- ceph .......................................................... [HEALTH_WARN]
  |   o- pools .......................................................... [Pools: 3]
' t8 V" ?% K% \; F7 h  |   | o- cephfs_data ... [(x3), Commit: 0.00Y/2028052096K (0%), Used: 2029431878b]
  |   | o- cephfs_metadata .... [(x3), Commit: 0.00Y/2028052096K (0%), Used: 77834b]
  |   | o- rbd ................ [(x3), Commit: 200G/2028052096K (10%), Used: 15352b]
( Z* f1 p1 F! n9 ?# C  |   o- topology ................................................ [OSDs: 6,MONs: 3]
  o- disks ........................................................ [200G, Disks: 1]
" I" |% C6 Y* ?5 o& X  | o- rbd ............................................................ [rbd (200G)]
  |   o- disk_1 ................................................ [rbd/disk_1 (200G)]
. Y/ S8 N: b$ e: \! N  o- iscsi-targets ............................... [DiscoveryAuth: None, Targets: 1]
    o- iqn.2003-01.com.redhat.iscsi-gw:iscsi-igw ..................... [Gateways: 2]
; u; G* C$ y* t% d      o- disks .......................................................... [Disks: 1]
5 R& S) S* {" A* X$ i; E6 `      | o- rbd/disk_1 ............................................... [Owner: node3]
      o- gateways ............................................ [Up: 2/2, Portals: 2]
      | o- node2 ............................................. [192.168.90.234 (UP)]
      | o- node3 ............................................. [192.168.90.235 (UP)]
7 x( t5 E! h5 D& Z0 U      o- host-groups .................................................. [Groups : 0]
      o- hosts .............................................. [Hosts: 1: Auth: CHAP]
        o- iqn.1994-05.com.redhat:rh7-client .......... [Auth: CHAP, Disks: 1(200G)]
% b2 [5 H& M; e: b+ z% t; x4 C          o- lun 0 ................................ [rbd/disk_1(200G), Owner: node3]
linux客户端配置
在作为客户端的linux主机上。

安装相关组件：
7 U, N; n7 h; p% n0 ^
sudo yum install -y iscsi-initiator-utils
sudo yum install -y device-mapper-multipath
. Y! q2 ^0 J  y% K" g$ Y开启multipathd服务并进行配置：

* W0 s. x6 S# \% |; p sudo mpathconf --enable --with_multipathd y
8 J  B( {( s! i3 S! N: ^9 U
- e- j$ R! e8 r8 Q2 @  y& r sudo vim /etc/multipath.conf
. y2 Q& A  z$ T2 O6 x sudo cat /etc/multipath.conf

; C8 c/ u) J& s) ^, }( k* wdevices {
/ [; U! p" e, J; N, A) ?        device {
: x8 e. L3 P3 l0 i: j: ]- }                vendor                 "LIO-ORG"
" D: R0 x" N' G  O4 A                hardware_handler       "1 alua"
. Z4 e  K2 H5 ]5 z                path_grouping_policy   "failover"
                path_selector          "queue-length 0"
! s& J" b8 b9 h5 o3 Q- a                failback               60
                path_checker           tur
                prio                   alua
                prio_args              exclusive_pref_bit
                fast_io_fail_tmo       25
1 }8 |. f0 z! {! I' P                no_path_retry          queue
' s. [4 `) x* ?! K0 w, m( Q* b1 r        }
9 m! G1 L; i* V: ~}
6 h: G/ d9 P, J修改客户端名称：
0 n+ t% b" [+ n4 c& R* z" N5 L
+ ^  c$ e+ F; m  x7 c" b9 c sudo vim /etc/iscsi/initiatorname.iscsi
! X# Y5 E$ a) k7 w; L sudo cat /etc/iscsi/initiatorname.iscsi

InitiatorName=iqn.1994-05.com.redhat:rh7-client
修改chap认证配置文件：
0 P: H( r' v; n, L
6 O- C# H- p2 `( E* T' r  Q9 `2 E sudo vim /etc/iscsi/iscsid.conf
6 A" @& \5 E, k9 V% O+ x9 `5 X3 Y sudo cat /etc/iscsi/iscsid.conf

...
*************
9 v# k% {. k- g& t/ M# f3 q CHAP Settings
*************
7 y* d5 N* c6 {5 n
To enable CHAP authentication set node.session.auth.authmethod
6 N$ Z+ E4 V3 a8 v7 S% ] to CHAP. The default is None.
node.session.auth.authmethod = CHAP

To set a CHAP username and password for initiator
8 h8 D: o1 t9 A% T+ X authentication by the target(s), uncomment the following lines:
node.session.auth.username = myiscsiusername
node.session.auth.password = myiscsipassword
...
- q. c7 s3 t. k% m" B/ u
, s7 A' Q  o) K  t  m" \1 w发现target：
/ Y1 {) H( g. b# H, w+ y+ O
sudo iscsiadm -m discovery -t st -p 192.168.90.234
; p- l: D% ]6 f8 s- S
192.168.90.234:3260,1 iqn.2003-01.org.linux-iscsi.rheln1
192.168.90.235:3260,2 iqn.2003-01.org.linux-iscsi.rheln1
登入target：

sudo iscsiadm -m node -T iqn.2003-01.org.linux-iscsi.rheln1 -l
( c' j4 J4 X, S. d& P4 v7 S, u8 i2 f查看是否成功：
2 f6 e  s( _% m1 |) j2 o( [
sudo multipath -ll
+ B9 z- v- d) ?7 F0 X0 xmpatha (360014050fedd563975249adb2e84e978) dm-2 LIO-ORG ,TCMU device     
size=200G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
: A+ L0 {- N# K7 L& D' r6 ~|-+- policy='queue-length 0' prio=50 status=active
9 l9 b. l& G8 j1 F/ x5 o| `- 3:0:0:0 sdc 8:32 active ready running
`-+- policy='queue-length 0' prio=10 status=enabled
  `- 2:0:0:0 sdb 8:16 active ready running
在fdisk中就可以直接看到这个“硬盘”：

7 B/ X0 W1 m9 ?' |) Y7 }9 e/ o sudo fdisk -l

...
, N0 t4 v3 d  |% \' U% kDisk /dev/mapper/mpatha: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
" v' @/ |, R6 f- C! ESector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 524288 bytes