openstack对接ceph存储

admin

As a storage administrator, you must install and configure Ceph before the Red Hat OpenStack Platform can use the Ceph block devices.

2.1. Prerequisites

• A new or existing Red Hat Ceph Storage cluster.
  ' _; {' P' D* s' h! U

( o8 |* C6 ~( m. i% W4 K5 H2.2. Creating Ceph pools for Openstack
! w6 o1 T0 `3 }- U2 m' u

Creating Ceph pools for use with OpenStack. By default, Ceph block devices use the rbdpool, but you can use any available pool.

Prerequisites

• A running Red Hat Ceph Storage cluster.
  & {4 {. J- p% t/ c

Procedure

• Verify the Red Hat Ceph Storage cluster is running, and is in a HEALTH_OK state:
  [root@mon ~]# ceph -s
• Create the Ceph pools:
  [size=1.12]Example
  
  [root@mon ~]# ceph osd pool create volumes 128[root@mon ~]# ceph osd pool create backups 128
  * v6 M' b6 Z9 h2 U# ?8 p

$ \' O4 S; H/ L# K7 |      [root@mon ~]# ceph osd pool create images 128[root@mon ~]# ceph osd pool create vms 128

6 j% b/ \4 s/ k0 R

In the above example, 128 is the number of placement groups.

IMPORTANT

- l  \% e! j' l* j! U1 G# J$ ~- f; B

Red Hat recommends using the Ceph Placement Group’s per Pool Calculator to calculate a suitable number of placement groups for the pools.

) f+ p' M! Q! g7 l
8 |( _( A- ?! H  j

Additional Resources

• See the Pools chapter in the Storage Strategies guide for more details on creating pools.
  

& T4 n1 \+ Z3 J2.3. Installing the Ceph client on Openstack
  c* z4 D, t$ S/ B( ?) V& M+ I* n" [
; n$ S" S. @  @7 P, u
7 ]; o3 {4 W9 y- z- f( p9 \5 m

Install the Ceph client packages on the Red Hat OpenStack Platform to access the Ceph storage cluster.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes.
  

Procedure

• On the OpenStack Nova, Cinder, Cinder Backup nodes install the following packages:
  [root@nova ~]# yum install python-rbd
  

      [root@nova ~]# yum install ceph-common
* x! O! ?. f% p% r% u5 X

• On the OpenStack Glance node install the python-rbd package:
  [root@glance ~]# yum install python-rbd
  5 |; @! M' T! F  z. N+ d* |2 O1 O

5 b6 }9 `& Q3 F% i8 F6 d: P2.4. Copying the Ceph configuration file to Openstack
- W- T0 c: F! R* a2 N
6 x' p  Z6 D. |( F4 _2 `( y8 W6 u! ^
- |8 W& c$ t* v! l! g

Copying the Ceph configuration file to the nova-compute, cinder-backup, cinder-volume, and glance-api nodes.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, and Glance nodes.
  * W8 M5 M3 i- z! Q$ {; i2 V  W0 E- t

+ l; r9 o6 e1 M2 Z1 Z

Procedure

• Copy the Ceph configuration file from the Ceph Monitor node to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes:
  [root@mon ~]# scp /etc/ceph/ceph.conf OPENSTACK_NODES:/etc/ceph
  

$ u: f: P  o; B8 \4 R  L2.5. Configuring Ceph client authentication

7 w; |0 n  m+ O. l4 ]: ?

Configure authentication for the Ceph client to access the Red Hat OpenStack Platform.

Prerequisites

• Root-level access to the Ceph Monitor node.
• A running Red Hat Ceph Storage cluster.
    B1 m  G" b& T8 Y' f( a) W

1 S3 ~% S- i8 s8 V

Procedure

• From a Ceph Monitor node, create new users for Cinder, Cinder Backup and Glance:
  [root@mon ~]# ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
    [' Z( C) V* A4 E" t

      [root@mon ~]# ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups'
      [root@mon ~]# ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'

• Add the keyrings for client.cinder, client.cinder-backup and client.glance to the appropriate nodes and change their ownership:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh CINDER_VOLUME_NODE sudo tee /etc/ceph/ceph.client.cinder.keyring
  7 v0 N" H8 e9 E( v1 J0 X

      [root@mon ~]# ssh CINDER_VOLUME_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
      [root@mon ~]# ceph auth get-or-create client.cinder-backup | ssh CINDER_BACKUP_NODE tee /etc/ceph/ceph.client.cinder-backup.keyring
( R: V4 _4 x9 ]; o0 z      [root@mon ~]# ssh CINDER_BACKUP_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring
      [root@mon ~]# ceph auth get-or-create client.glance | ssh GLANCE_API_NODE sudo tee /etc/ceph/ceph.client.glance.keyring
      [root@mon ~]# ssh GLANCE_API_NODE chown glance:glance /etc/ceph/ceph.client.glance.keyring

• OpenStack Nova nodes need the keyring file for the nova-compute process:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh NOVA_NODE tee /etc/ceph/ceph.client.cinder.keyring
• The OpenStack Nova nodes also need to store the secret key of the client.cinderuser in libvirt. The libvirt process needs the secret key to access the cluster while attaching a block device from Cinder. Create a temporary copy of the secret key on the OpenStack Nova nodes:
  [root@mon ~]# ceph auth get-key client.cinder | ssh NOVA_NODE tee client.cinder.key
  If the storage cluster contains Ceph block device images that use the exclusive-lockfeature, ensure that all Ceph block device users have permissions to blacklist clients:
  [root@mon ~]# ceph auth caps client.ID mon 'allow r, allow command "osd blacklist"' osd 'EXISTING_OSD_USER_CAPS'
• Return to the OpenStack Nova node:
  [root@mon ~]# ssh NOVA_NODE
• Generate a UUID for the secret, and save the UUID of the secret for configuring nova-compute later:
  [root@nova ~]# uuidgen > uuid-secret.txtNOTE
  9 P8 y6 X0 ~0 x8 ?3 r4 `5 U2 V+ u$ Q( E+ T
  You do not necessarily need the UUID on all the Nova compute nodes. However, from a platform consistency perspective, it’s better to keep the same UUID.
• On the OpenStack Nova nodes, add the secret key to libvirt and remove the temporary copy of the key:
  cat > secret.xml <<EOF<secret ephemeral='no' private='no'>  <uuid>`cat uuid-secret.txt`</uuid>  <usage type='ceph'>    <name>client.cinder secret</name>  </usage></secret>EOF
• Set and define the secret for libvirt:
  [root@nova ~]# virsh secret-define --file secret.xml[root@nova ~]# virsh secret-set-value --secret $(cat uuid-secret.txt) --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml
  , T, y7 e3 P) m& o

Additional Resources