openstack对接ceph存储

admin

As a storage administrator, you must install and configure Ceph before the Red Hat OpenStack Platform can use the Ceph block devices.

2.1. Prerequisites

8 |# |/ X& G4 }. {, B$ ~
/ o, w8 |" q0 l

• A new or existing Red Hat Ceph Storage cluster.
  

+ j% @3 S4 M: A( r2 h2.2. Creating Ceph pools for Openstack

& E, C: F- s/ O3 u8 |! [9 _# Y; c
9 H2 j" c' \& x9 c+ ^1 {

Creating Ceph pools for use with OpenStack. By default, Ceph block devices use the rbdpool, but you can use any available pool.

Prerequisites

• A running Red Hat Ceph Storage cluster.
  5 N+ ^8 k. d" T5 Z$ s

Procedure

• Verify the Red Hat Ceph Storage cluster is running, and is in a HEALTH_OK state:
  [root@mon ~]# ceph -s
• Create the Ceph pools:
  [size=1.12]Example
  
  [root@mon ~]# ceph osd pool create volumes 128[root@mon ~]# ceph osd pool create backups 128
  

      [root@mon ~]# ceph osd pool create images 128[root@mon ~]# ceph osd pool create vms 128
; v$ Y; ]$ H9 v. p" R* j
. x6 \# G' n% ]0 p1 G6 e

In the above example, 128 is the number of placement groups.

IMPORTANT
" u, Q: F) e: \+ S+ X/ S

Red Hat recommends using the Ceph Placement Group’s per Pool Calculator to calculate a suitable number of placement groups for the pools.

3 O3 Y$ w1 ^2 F2 C0 m
+ _6 L/ L5 g, Z0 l0 ~

Additional Resources

• See the Pools chapter in the Storage Strategies guide for more details on creating pools.
  * w* i: v' x. ~1 }# L; s

5 |: H6 L, u* L; `* m: c2.3. Installing the Ceph client on Openstack

Install the Ceph client packages on the Red Hat OpenStack Platform to access the Ceph storage cluster.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes.
  * l' B0 _& b( T% L- T

Procedure

• On the OpenStack Nova, Cinder, Cinder Backup nodes install the following packages:
  [root@nova ~]# yum install python-rbd
  

      [root@nova ~]# yum install ceph-common

• On the OpenStack Glance node install the python-rbd package:
  [root@glance ~]# yum install python-rbd
  

2.4. Copying the Ceph configuration file to Openstack

Copying the Ceph configuration file to the nova-compute, cinder-backup, cinder-volume, and glance-api nodes.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, and Glance nodes.
  ) I4 o% R% r; c9 S' P

Procedure

• Copy the Ceph configuration file from the Ceph Monitor node to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes:
  [root@mon ~]# scp /etc/ceph/ceph.conf OPENSTACK_NODES:/etc/ceph
  

2.5. Configuring Ceph client authentication
7 v6 _1 u+ ^3 q* P- i8 a- [( ^" W

, l; C: Y& J0 S) J" R$ ^

Configure authentication for the Ceph client to access the Red Hat OpenStack Platform.

Prerequisites

• Root-level access to the Ceph Monitor node.
• A running Red Hat Ceph Storage cluster.
  

Procedure

• From a Ceph Monitor node, create new users for Cinder, Cinder Backup and Glance:
  [root@mon ~]# ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
  

      [root@mon ~]# ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups'
      [root@mon ~]# ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'
$ n  S& _5 h" {% }

• Add the keyrings for client.cinder, client.cinder-backup and client.glance to the appropriate nodes and change their ownership:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh CINDER_VOLUME_NODE sudo tee /etc/ceph/ceph.client.cinder.keyring
  $ T9 C, X; G3 a1 x- I

      [root@mon ~]# ssh CINDER_VOLUME_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
, c7 i1 R- p' z' }6 @& [  ?      [root@mon ~]# ceph auth get-or-create client.cinder-backup | ssh CINDER_BACKUP_NODE tee /etc/ceph/ceph.client.cinder-backup.keyring
      [root@mon ~]# ssh CINDER_BACKUP_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring
      [root@mon ~]# ceph auth get-or-create client.glance | ssh GLANCE_API_NODE sudo tee /etc/ceph/ceph.client.glance.keyring
8 Q0 B% z1 g) @5 N  L" c) E      [root@mon ~]# ssh GLANCE_API_NODE chown glance:glance /etc/ceph/ceph.client.glance.keyring
( K1 E, ^, M2 |' `- O+ i& C

• OpenStack Nova nodes need the keyring file for the nova-compute process:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh NOVA_NODE tee /etc/ceph/ceph.client.cinder.keyring
• The OpenStack Nova nodes also need to store the secret key of the client.cinderuser in libvirt. The libvirt process needs the secret key to access the cluster while attaching a block device from Cinder. Create a temporary copy of the secret key on the OpenStack Nova nodes:
  [root@mon ~]# ceph auth get-key client.cinder | ssh NOVA_NODE tee client.cinder.key
  If the storage cluster contains Ceph block device images that use the exclusive-lockfeature, ensure that all Ceph block device users have permissions to blacklist clients:
  [root@mon ~]# ceph auth caps client.ID mon 'allow r, allow command "osd blacklist"' osd 'EXISTING_OSD_USER_CAPS'
• Return to the OpenStack Nova node:
  [root@mon ~]# ssh NOVA_NODE
• Generate a UUID for the secret, and save the UUID of the secret for configuring nova-compute later:
  [root@nova ~]# uuidgen > uuid-secret.txtNOTE
  
  You do not necessarily need the UUID on all the Nova compute nodes. However, from a platform consistency perspective, it’s better to keep the same UUID.
• On the OpenStack Nova nodes, add the secret key to libvirt and remove the temporary copy of the key:
  cat > secret.xml <<EOF<secret ephemeral='no' private='no'>  <uuid>`cat uuid-secret.txt`</uuid>  <usage type='ceph'>    <name>client.cinder secret</name>  </usage></secret>EOF
• Set and define the secret for libvirt:
  [root@nova ~]# virsh secret-define --file secret.xml[root@nova ~]# virsh secret-set-value --secret $(cat uuid-secret.txt) --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml
  

Additional Resources

8 D' I4 p) V( v! w: A2 ?