Install and Configure a Simple Ceph Object Gateway

admin

Install and Set Up the Ceph Object Gateway

; X, z3 [' l% l1 AInstall Ceph Object Gateway Daemon
Install and configure the Ceph Object Gateway daemon on the gateway host:

! b; I& ^- r8 \! m4 `( f6 D# yum install ceph-radosgw
( n0 }( P, {7 ^4 \  D6 g6 }Note that for federated architectures, you would additionally install the synchronization agent, radosgw-agent, to handle the metadata synchronization between zones and regions.
4 X4 n& m# R/ I$ n: o
Additional Basic Requirements To Run The Object Gateway Daemon
; o6 z1 x; l* n+ S( O7 R  K7 ^- A5 KSome actions are not performed automatically during the installation of the Ceph Object Gateway daemon, since these steps may vary if you are configuring a federated gateway or you have chosen an alternative deployment approach. If you have followed the instructions provided so far, continue by performing the following steps:

Create the Ceph Object Gateway data directory manually, if it does not already exist:
' q* W6 i5 V. G) A
& D' i& N! Z# Z2 z: b, D# mkdir -p /var/lib/ceph/radosgw/ceph-radosgw.gateway
Update the ownership of the socket directory to allow the Object Gateway daemon to write to it. The daemon runs as the unprivileged Apache UID. To grant permissions to the default socket location, do the following on the gateway host:

9 H4 j1 S- ?, C: k3 ~- l0 s" R! T# chown apache:apache /var/run/ceph
The root user owns the log file, by default, but since the Object Gateway daemon runs as the unprivileged Apache UID, ownership of this file must be changed to allow the Object Gateway to write to it. Do the following on the gateway host:

# chown apache:apache /var/log/radosgw/client.radosgw.gateway.log
Create a User and Keyring To Authenticate The Object Gateway To The Ceph Storage Cluster
# {+ H7 w5 v8 pThe Ceph Object Gateway must have a user name and key to communicate with a Ceph Storage Cluster. In the following steps, an admin node in the Ceph Storage Cluster is used to create a keyring. A client user name and key is then created for the Ceph Object Gateway. The key is added to the Ceph Storage Cluster. Finally, the keyring is copied to the node running the Ceph Object Gateway, so that it can use it to access the Ceph Storage Cluster.

$ y; q1 g3 r) O& T5 C! WExecute the following steps on the admin node of your cluster:

6 c- m9 f8 b  ~; q) S2 ~; x) ICreate a keyring for the gateway:
) F& d. p& K, K
# ceph-authtool --create-keyring /etc/ceph/ceph.client.radosgw.keyring
* V/ A. h- X; B5 @/ L# chmod +rw /etc/ceph/ceph.client.radosgw.keyring
Generate a Ceph Object Gateway user name and key and add it to the keyring:

0 R8 _4 Z2 _; L& X* `# u) s# ceph-authtool /etc/ceph/ceph.client.radosgw.keyring -n client.radosgw.gateway --gen-key
Note that the Ceph Object Gateway user name is set to client.radosgw.gateway.

# h7 m5 T# s) LAdd capabilities to the key:

# ceph-authtool -n client.radosgw.gateway --cap osd 'allow rwx' --cap mon 'allow rwx' /etc/ceph/ceph.client.radosgw.keyring
Add the key to the Ceph Storage Cluster to enable the Ceph Object Gateway access:
% ?' y8 G7 N# d
# ceph -k /etc/ceph/ceph.client.admin.keyring auth add client.radosgw.gateway -i /etc/ceph/ceph.client.radosgw.keyring
If the admin node of the cluster is not also used to host the Object Gateway, you must copy the keyring to the gateway host. Typically, this is done using scp, for example:

3 b* Q( Y/ c) Y: j" `8 E6 P# scp /etc/ceph/ceph.client.radosgw.keyring user@gw.example.com:
On the gateway host, you must ensure that the keyring is moved to the correct location, at /etc/ceph/ceph.client.radosgw.keyring:
$ q( q/ K; {$ m8 d. f5 p  m
; [$ k4 ^: n' E8 s6 a# mv ceph.client.radosgw.keyring /etc/ceph/ceph.client.radosgw.keyring
5 Y0 K5 i- A3 v% \8 ^1 n- N- PCreate Pools for the Object Gateway
Ceph Object Gateways require Ceph Storage Cluster pools to store specific gateway data. If the user you created has permissions, the gateway daemon creates the pools automatically. If you intend to allow the gateway daemon to create pools automatically, you should ensure that an appropriate default number of placement groups per pool is set in your Ceph configuration file (/etc/ceph/ceph.conf) on the admin node. Ceph Object Gateways have multiple pools, so you should keep the default number of placement groups per pool low to maximize performance.

You can manually create the pools if you wish to use alternative values for the number of placement groups per pool. The default pool names for an Object Gateway are as follows:
! v3 Y. t' G% L
5 I2 Y1 N  x. R. h3 Z0 C8 c1 M.rgw.root

- P, y0 Y. I* H1 i* @.rgw.control

.rgw.gc
7 K! X2 G+ P( ?1 ^! z1 l
.rgw.buckets

* x# f6 q2 N, |/ t$ f6 n, a4 X.rgw.buckets.index

.log

  r0 S& e5 f6 O5 v.intent-log
9 i- z5 e5 {- u: L. U. E
.usage

  e0 v7 M( ?1 k.users
2 Z$ I6 [7 q7 Z1 N
) l5 m, \. b: ]1 ?.users.email
6 ]  e9 m6 \( p* M
( p8 S5 n3 i+ X6 Q* R.users.swift

.users.uid

Use the following command to manually create each of these pools, substituting poolname with the name of the pool that you are creating, pg-num with the number of placement groups to create for the pool, and pgp-num for the number of placement groups for placement (usually the same or greater than the number of placement groups).

3 K1 d0 Y) L! ~# ceph osd pool create poolname pg-num pgp-num
7 V  e4 {3 N4 A% o: {You can list the available pools and check that pools have been created for the gateway by running:

& m8 P5 ]7 k. Q7 x, Y4 F7 a# rados lspools
4 {# s4 r) b! B  y) TNote that if you have not yet started the gateway daemon, only pools that you have manually created are listed at this point.
8 u3 _! D& M& y* p
Add the Ceph Gateway Configuration Details To The Ceph Configuration File
6 o9 K8 q* Z/ XDetails of the Ceph Object Gateway configuration must be made available to the Ceph Storage Cluster. Edit the Ceph Configuration file on the admin node of the cluster. Create a configuration entry that identifies the Ceph Object Gateway instance, provides the short hostname of the gateway host, provides a path to the keyring file, provides a path to a lock file and specifies the socket information for FastCGI. This entry is slightly different depending on whether you are running Ceph on Oracle Linux 6 or on Oracle Linux 7, since Oracle Linux 6 uses localhost TCP for the FastCGI socket, while Oracle Linux 7 uses Unix Domain Sockets.
1 I/ ^  J' o- k3 W9 r
3 k0 F2 a3 H2 iFor Oracle Linux 6, append the following configuration to /etc/ceph/ceph.conf on the admin node of the Ceph Storage Cluster:

' b  ]! E+ L( P% Z' A7 v, _/ x+ {[client.radosgw.gateway]
: g5 r0 m, Y$ w, Thost = hostname
keyring = /etc/ceph/ceph.client.radosgw.keyring
  `! N- h9 Q: X9 Ergw socket path = ""
. }2 j0 y9 g2 e" |  k, wlog file = /var/log/radosgw/client.radosgw.gateway.log
4 [3 I1 \; f4 S3 c6 f. R) trgw frontends = fastcgi socket_port=9000 socket_host=0.0.0.0
9 A4 L; n3 [2 O) PFor Oracle Linux 7, append the following configuration to /etc/ceph/ceph.conf on the admin node of the Ceph Storage Cluster:

[client.radosgw.gateway]
' G  y8 F$ V: g2 [host = hostname
, ^7 r9 ]$ n3 ^0 r1 r4 @0 ?keyring = /etc/ceph/ceph.client.radosgw.keyring
rgw socket path = /var/run/ceph/ceph.radosgw.gateway.fastcgi.sock
log file = /var/log/radosgw/client.radosgw.gateway.log
rgw print continue = false
+ w- D/ @* j. I- w/ mReplace hostname with the short hostname of the gateway host. You can obtain this value by running hostname -s on the gateway host.
6 M( Q% v1 [+ t6 k2 i8 V- N
4 S3 k, w6 o0 W- a  k( P: sUpdate Ceph Configuration Across All Cluster Nodes
The updated Ceph configuration file must be copied to all Ceph cluster nodes. This is achieved using the ceph-deploy command on the admin node of the cluster.

) P0 b2 t3 g) P% k) @! {# eFirst, copy the ceph.conf file to the root directory used by the cluster on the admin node:
5 i. ~7 `+ G; A! E( k
& n  Y; l6 M$ P# cp /etc/ceph/ceph.conf /var/mydom_ceph
% w: \' S- V6 E$ @+ _$ K3 W% cNext, pull the configuration from the cluster directory into the admin node:
) E' M" m8 ]* z, f& Y2 j; j
$ d0 E  n( O$ y5 b# ceph-deploy --overwrite-conf config pull hostname
+ R2 a" e" i' n2 I/ q/ m+ DSubstitute hostname with the short hostname of the Ceph admin node. You can obtain this value by running hostname -s on the admin node. These commands cause the contents of the ceph.conf file to be overwritten.

Finally, push the updated configuration from the admin node to all other nodes in the cluster including the gateway host. Run the following command for each host in the cluster:
1 p& V; y, C+ C0 \) z" ?
  n4 K& G6 f0 n$ L" q9 U3 A( h5 E# ceph-deploy --overwrite-conf config push hostname
+ d2 o# s, _3 Q6 ?Substitute hostname with the short hostname of each node in the cluster, including the gateway host. You may run this as a single command by substituting hostname with a space-separated list of all of the hostnames that you wish to push the configuration update to.
& o2 X" X+ l/ J& l* {
Create a CGI wrapper script
% W+ q  I3 x, A/ Z2 YA wrapper CGI script is used to provide an interface between Apache and the Object Gateway daemon. You must create the script yourself in a location that is accessible to Apache on the gateway host.
9 `: A( P- a) ^
Create /var/www/html/s3gw.fcgi and open it in an editor. Add the following content to the file:

#!/bin/sh
exec /usr/bin/radosgw -c /etc/ceph/ceph.conf -n client.radosgw.gateway
, O! R0 n( a4 o6 F, h6 k# MProvide execute permissions to the script and update ownership so that Apache is able to execute the script:
, P* I8 u7 `6 Y  O+ f$ K
! H  w2 U3 s4 O; E# chmod +x /var/www/html/s3gw.fcgi
# chown apache:apache /var/www/html/s3gw.fcgi
Start The Object Gateway Daemon
Start the Ceph Object Gateway daemon on the gateway host.
5 D! A( x% L# v/ [$ E( x$ l
On Oracle Linux 6, run:
; u, \4 F% Z! X; r, t7 ~2 V
# service ceph-radosgw start
8 u/ T3 ~0 f$ I3 f$ w# chkconfig ceph-radosgw on
! H! _- F2 v8 }8 p3 Q7 R2 N! uOn Oracle Linux 7, run:

# systemctl start ceph-radosgw
# chkconfig ceph-radosgw on
; ]7 w* y. ~6 A6 HCreate An Apache VirtualHost Entry For the Ceph Object Gateway
Apache must be configured to provide access to the FastCGI wrapper script so that the Ceph Object Gateway is able to function. To do this, create a VirtualHost entry in your Apache configuration. It is best to create this as an individual configuration file in /etc/httpd/conf.d. The VirtualHost entry differs slightly depending on the version of Oracle Linux you are using. Instructions are provided for both.

On Oracle Linux 6, create the file /etc/httpd/conf.d/rgw.conf and add the following content:
0 B* N+ _& v" I- M8 C
<VirtualHost 198.51.100.1:80>
0 k  ^( G$ H" ~% S: ]5 LServerName gw.example.com
DocumentRoot /var/www/html
2 \9 T" D5 Y5 l% y3 a: FErrorLog /var/log/httpd/rgw_error.log
& H0 o$ H$ x4 {# kCustomLog /var/log/httpd/rgw_access.log combined
RewriteEngine On
0 l' |$ E1 p& M$ F5 DRewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
. D, g8 z/ V0 Z: H9 X$ ?9 XSetEnv proxy-nokeepalive 1
% E! g4 [1 z7 x7 |' PProxyPass / fcgi://localhost:9000/
8 q. T) j5 D' ?5 w( u( n' Y6 {</VirtualHost>
where 198.51.100.1 must be replaced with the public facing IP address of the host. Substitute gw.example.com so that the ServerName directive points to the hostname or fully qualified domain name of the gateway host.
+ V0 @  Q( X# t- O
On Oracle Linux 7, create the file /etc/httpd/conf.d/rgw.conf and add the following content:
/ q- E/ Y& S8 M! y+ ~6 B
<VirtualHost 198.51.100.1:80>
ServerName gw.example.com
5 _2 E: x$ o% O) d/ s( cDocumentRoot /var/www/html
ErrorLog /var/log/httpd/rgw_error.log
CustomLog /var/log/httpd/rgw_access.log combined
RewriteEngine On
, C5 _4 G+ @. ?2 I6 B( O6 ?2 [3 t1 IRewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization},L]
  }4 Y' P' h6 K9 B+ C. w  b( ySetEnv proxy-nokeepalive 1
" K+ A* ?1 O0 g/ j; CProxyPass / unix:///var/run/ceph/ceph.radosgw.gateway.fastcgi.sock|fcgi://localhost:9000/
1 \( v8 Y6 ?  Y</VirtualHost>
where 198.51.100.1 must be replaced with the public facing IP address of the host. Substitute gw.example.com so that the ServerName directive points to the hostname or fully qualified domain name of the gateway host.
# L- ]/ X9 _1 q7 x
Restart Apache
The httpd service needs to be restarted to use the new configuration.
) U% q) c$ c  O- J& v5 B3 B6 M
, q$ ~2 q+ o. o! M; d+ r" `! BOn Oracle Linux 6, do:

% p2 r+ i2 j5 I2 k" o  p# service httpd restart
9 `- W6 v" B8 R4 J$ }# H6 I. R; v# chkconfig httpd on
Alternately, on Oracle Linux 7, do:
3 a# N5 {/ X4 \" _4 [
# systemctl restart httpd
# systemctl enable httpd
; b4 U' ?, r2 J7 W5 g+ \At this point, the Ceph Object Gateway should be running and the REST interfaces available. You must create and initial Ceph Object Gateway user for the S3 interface and a subuser for the Swift interface.

  a9 k& r7 W- D% Y3 G( e4 r