openstack对接ceph存储

admin

As a storage administrator, you must install and configure Ceph before the Red Hat OpenStack Platform can use the Ceph block devices.

2.1. Prerequisites

• A new or existing Red Hat Ceph Storage cluster.
  

2.2. Creating Ceph pools for Openstack
" _. N3 C. O& A/ ?& }5 Z
3 P) E9 N+ T! n- J/ ?! v7 Y

Creating Ceph pools for use with OpenStack. By default, Ceph block devices use the rbdpool, but you can use any available pool.

Prerequisites

• A running Red Hat Ceph Storage cluster.
  , m8 W6 X$ G, X! w! J/ ?

. W, F* o1 N1 k% D" x; c) f

Procedure

• Verify the Red Hat Ceph Storage cluster is running, and is in a HEALTH_OK state:
  [root@mon ~]# ceph -s
• Create the Ceph pools:
  [size=1.12]Example
  
  [root@mon ~]# ceph osd pool create volumes 128[root@mon ~]# ceph osd pool create backups 128
  1 A- Z' c- r/ {8 E5 w1 }

      [root@mon ~]# ceph osd pool create images 128[root@mon ~]# ceph osd pool create vms 128


% n' _# V2 b( y$ K2 X

In the above example, 128 is the number of placement groups.

IMPORTANT
' ^0 @5 |1 F2 E$ U) l- s, W/ t

Red Hat recommends using the Ceph Placement Group’s per Pool Calculator to calculate a suitable number of placement groups for the pools.

Additional Resources

• See the Pools chapter in the Storage Strategies guide for more details on creating pools.
  

2.3. Installing the Ceph client on Openstack
# y1 K) P4 R# y- H
% D$ ?+ V# C; S! O
- }0 ^& i$ c6 M. d( P

Install the Ceph client packages on the Red Hat OpenStack Platform to access the Ceph storage cluster.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes.
  1 B8 [  T% _4 [& j' h

: _, [# k( _$ o, G

Procedure

• On the OpenStack Nova, Cinder, Cinder Backup nodes install the following packages:
  [root@nova ~]# yum install python-rbd
  

      [root@nova ~]# yum install ceph-common

• On the OpenStack Glance node install the python-rbd package:
  [root@glance ~]# yum install python-rbd
  

$ {3 @- E: t3 E. k/ i; f2.4. Copying the Ceph configuration file to Openstack
% \5 `0 G0 K# B- {; k
- T9 e4 N/ v: U2 T: ~* I' P
0 f$ Q; \. o5 @* m

Copying the Ceph configuration file to the nova-compute, cinder-backup, cinder-volume, and glance-api nodes.

Prerequisites

• A running Red Hat Ceph Storage cluster.
• Access to the Ceph software repository.
• Root-level access to the OpenStack Nova, Cinder, and Glance nodes.
  

6 O# g% R9 h( P1 P4 R% a  J

Procedure

• Copy the Ceph configuration file from the Ceph Monitor node to the OpenStack Nova, Cinder, Cinder Backup and Glance nodes:
  [root@mon ~]# scp /etc/ceph/ceph.conf OPENSTACK_NODES:/etc/ceph
  

2.5. Configuring Ceph client authentication


5 }8 |/ p2 j6 J5 b. i) C  F& G: p+ w& y

Configure authentication for the Ceph client to access the Red Hat OpenStack Platform.

Prerequisites

• Root-level access to the Ceph Monitor node.
• A running Red Hat Ceph Storage cluster.
  

Procedure

• From a Ceph Monitor node, create new users for Cinder, Cinder Backup and Glance:
  [root@mon ~]# ceph auth get-or-create client.cinder mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=volumes, allow rwx pool=vms, allow rx pool=images'
  

0 e6 }! `9 T* Q: k9 a8 X3 `      [root@mon ~]# ceph auth get-or-create client.cinder-backup mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=backups'
      [root@mon ~]# ceph auth get-or-create client.glance mon 'allow r' osd 'allow class-read object_prefix rbd_children, allow rwx pool=images'
  I4 M9 l' a) E, G( a8 H

• Add the keyrings for client.cinder, client.cinder-backup and client.glance to the appropriate nodes and change their ownership:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh CINDER_VOLUME_NODE sudo tee /etc/ceph/ceph.client.cinder.keyring
  

      [root@mon ~]# ssh CINDER_VOLUME_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder.keyring
      [root@mon ~]# ceph auth get-or-create client.cinder-backup | ssh CINDER_BACKUP_NODE tee /etc/ceph/ceph.client.cinder-backup.keyring
      [root@mon ~]# ssh CINDER_BACKUP_NODE chown cinder:cinder /etc/ceph/ceph.client.cinder-backup.keyring
      [root@mon ~]# ceph auth get-or-create client.glance | ssh GLANCE_API_NODE sudo tee /etc/ceph/ceph.client.glance.keyring
      [root@mon ~]# ssh GLANCE_API_NODE chown glance:glance /etc/ceph/ceph.client.glance.keyring

• OpenStack Nova nodes need the keyring file for the nova-compute process:
  [root@mon ~]# ceph auth get-or-create client.cinder | ssh NOVA_NODE tee /etc/ceph/ceph.client.cinder.keyring
• The OpenStack Nova nodes also need to store the secret key of the client.cinderuser in libvirt. The libvirt process needs the secret key to access the cluster while attaching a block device from Cinder. Create a temporary copy of the secret key on the OpenStack Nova nodes:
  [root@mon ~]# ceph auth get-key client.cinder | ssh NOVA_NODE tee client.cinder.key
  If the storage cluster contains Ceph block device images that use the exclusive-lockfeature, ensure that all Ceph block device users have permissions to blacklist clients:
  [root@mon ~]# ceph auth caps client.ID mon 'allow r, allow command "osd blacklist"' osd 'EXISTING_OSD_USER_CAPS'
• Return to the OpenStack Nova node:
  [root@mon ~]# ssh NOVA_NODE
• Generate a UUID for the secret, and save the UUID of the secret for configuring nova-compute later:
  [root@nova ~]# uuidgen > uuid-secret.txtNOTE
  ( Z+ `$ g! o( {, ?4 z9 ?
  You do not necessarily need the UUID on all the Nova compute nodes. However, from a platform consistency perspective, it’s better to keep the same UUID.
• On the OpenStack Nova nodes, add the secret key to libvirt and remove the temporary copy of the key:
  cat > secret.xml <<EOF<secret ephemeral='no' private='no'>  <uuid>`cat uuid-secret.txt`</uuid>  <usage type='ceph'>    <name>client.cinder secret</name>  </usage></secret>EOF
• Set and define the secret for libvirt:
  [root@nova ~]# virsh secret-define --file secret.xml[root@nova ~]# virsh secret-set-value --secret $(cat uuid-secret.txt) --base64 $(cat client.cinder.key) && rm client.cinder.key secret.xml
  $ V' E' d6 B6 O' c& B

0 L5 e* h' z2 B/ G; C9 H

Additional Resources

* e" N0 U$ E! b! O! o" C+ e