|
|
Ceph 对象网关主要是由 radosgw 接受用户请求，然后与后端librados交互;它主要起到承上启下的功能，对外它能兼容两种接口，一种是S3，另一种是Openstack Swift 接口，这两种接口，都有各自的用户认证机制，所以 Ceph 也提供了一套独立的用户管理机制，这套用户管理机制能同时兼容S3和Swift接口，最终数据会落在 OSD 上面，无论使用 S3 还是 Swift ，它们落到 OSD 上面，都是相同的名称空间;所以你使用 S3 存储的，也可以使用 Swift 进行访问。这是对象存储的一个基本架构，所以如果要使用它，我们需要部署radosgw，才能访问到集群，默认是没有安装的。6 u5 G0 z1 P" ^8 Y/ c3 r( g3 ~% r/ R+ u
; N! ^0 S% I* D: i$ k! ]bucket 是什么?可以理解为装载对象的容器，它的后端是无限可扩展的存储空间，并且具备安全可靠性。Ceph 对象存储它后端是借助 ceph rados 实现数据容灾的机制，那么他能提供哪些功能呢?
% P7 f: Y9 F/ A: A4 Q# F8 ?* t+ O3 [& M/ F& w$ ]
基本功能- L+ J+ Y: \, k0 S0 G* o, Y8 @' i
RESTful Interface # RESTful风格的接口，实现上传下载及管理功能;% u9 B7 S; U) N$ c6 m! t
S3- and Swift-compliant APIs # 提供两种风格的 API 接口，兼容 S3 和 Swift;
7 P# u, N! }8 R: a! }8 |S3-style subdomains
; S& y! Q/ _# D# o& @5 dUnified S3/Swift namespace # 扁平化、统一的的S3/Swift的名称空间;
! I! l( m+ b5 MUser management # 为了安全性，也提供用户管理，可以限制对象是可以公共访问，还是授权访问;
0 ^9 \) N# ~. P/ C5 e+ Y. [3 NUsage tracking # 追踪用户使用情况 rados df+ h/ _- z7 [+ k; P7 G
Striped objects # 支持分片上传
3 n) T4 m; k' W2 B, x$ m5 |Cloud solution integration # 支持云解决方案集成: @3 i$ i0 P& W' H
Multi-site deployment # 支持多站点部署
/ v6 u, l" X' t1 Z9 @Multi-site replication # 支持多站点复制& T" v0 C$ X- |/ P3 \' [& _7 k4 g
安装 RGW
; f( ?" o) e+ a/ |2 x* a V1. 软件安装
+ V1 z# u! ^+ a. q5 v, r6 n复制, u! d7 K9 J1 t! s& n8 ?
[root@ceph-node01 ~]# rpm -qa |grep ceph ' W+ N9 b- `+ e# A( @; n6 a1 }; I/ H
ceph-base-14.2.11-0.el7.x86_64
" ]. k8 u" O. P( qceph-mon-14.2.11-0.el7.x86_64 0 }/ y3 X) [/ C V1 e( r. V
ceph-deploy-2.0.1-0.noarch % ~" c# P, g; {+ o- u/ O0 X
python-ceph-argparse-14.2.11-0.el7.x86_64
9 G: x* P* E! @0 h1 b7 t7 o5 nlibcephfs2-14.2.11-0.el7.x86_64
" |2 U/ L; F3 m8 W8 _ceph-common-14.2.11-0.el7.x86_64 5 a( v' X& S0 s
ceph-selinux-14.2.11-0.el7.x86_64 % U6 W+ Z ~' D
ceph-mds-14.2.11-0.el7.x86_64
8 ?* \; F0 c6 X6 C& B4 g+ \ceph-14.2.11-0.el7.x86_64 5 R p" H' k: R
python-cephfs-14.2.11-0.el7.x86_64
: x4 ?! L6 u5 Z, T: |ceph-osd-14.2.11-0.el7.x86_64 ) p4 C! t! b# |9 B/ E
ceph-mgr-14.2.11-0.el7.x86_64 3 j; g# G5 N$ k: Y
ceph-radosgw-14.2.11-0.el7.x86_64 # 直接使用 yum 安装即可
+ p) K8 `2 n4 \4 g/ N; c% r- F[root@ceph-node01 ~]# 6 t( }: }$ S1 W( C
0 @; ~2 |) D( I" c$ m9 @
H+ ]5 a; p) F6 x. V+ Q' z2. 启动服务，默认启动在7480端口8 C9 u; W% A p+ p( }" s# V
复制6 D0 L6 m7 p! |' X$ Q/ N( s3 x' C# [3 K
[root@ compute01 ceph-deploy]# ceph-deploy rgw create compute01
+ w3 N% T& @& t9 z' O$ f1 w4 d% m$ O( J- a: m+ a" L" O
3. 服务检测
& \0 A2 t5 [+ z, t3 W: v复制
" Z5 w9 W. Y( k[root@ compute01 ceph-deploy]# systemctl status ceph-radosgw@rgw.compute01 4 ?3 E1 _4 y5 h/ G
● ceph-radosgw@rgw. compute01.service - Ceph rados gateway
% N( e d# i; V& _! N% D# q Loaded: loaded (/usr/lib/systemd/system/ceph-radosgw@.service; enabled; vendor preset: disabled) ' y- n! e7 _. o: z1 e& i. u
Active: active (running) since 一 2020-10-05 20:34:36 EDT; 14s ago
& ]$ k1 J8 P, ~6 ~5 r' x Main PID: 33574 (radosgw)
2 T2 ~ {( t- V. O0 ]! L CGroup: /system.slice/system-ceph\x2dradosgw.slice/ceph-radosgw@rgw.compute01.service 2 H! v. q1 W; f* @4 U
└─33574 /usr/bin/radosgw -f --cluster ceph --name client.rgw.ceph-node01 --setuser ceph --setgroup ceph 1 z; [2 I1 ?. o+ @
" t7 D6 T2 `/ ?* |10月 05 20:34:36 compute01 systemd[1]: Started Ceph rados gateway.
, @/ u% X! n3 { D) i10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:13] Unknown lvalue 'LockPersonality' in section 'Service' $ A( C* a6 f- V# j: O8 r
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:14] Unknown lvalue 'MemoryDenyWriteExecute' in ...Service'
5 J( L/ J" g' ^- J8 o) w10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:17] Unknown lvalue 'ProtectControlGroups' in se...Service'
% p" E( Q+ `8 M5 @7 a; X8 ~8 w10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:19] Unknown lvalue 'ProtectKernelModules' in se...Service' R( ~8 b+ r5 T- g8 e7 w( y( r
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:20] Unknown lvalue 'ProtectKernelTunables' in s...Service' 2 B3 B7 v; `- j7 p) f% g- \
Hint: Some lines were ellipsized, use -l to show in full. * C }2 |6 I: |8 m0 p% H
[root@ceph-node01 ceph-deploy]# netstat -antp |grep 7480
9 H$ B& y% L2 \ ftcp 0 0 0.0.0.0:7480 0.0.0.0:* LISTEN 33574/radosgw
: n9 W1 i; K, B/ O/ u[root@ceph-node01 ceph-deploy]# ceph -s " a) }7 |. i$ b) P
cluster:
/ R- l% W0 H& B, q; k" H id: a6d0359e-8295-4e62-8620-54273462e79f
, g* q }' S1 J) h% p health: HEALTH_OK3 F; X$ [. P# u5 X& f0 Z
$ d/ r! ?9 j/ c$ Y$ q
services:
4 w/ | a; v8 j2 ^ mon: 3 daemons, quorum compute01,compute02,compute03 (age 12d)+ i3 J E, Q2 e+ i: M1 v; i
mgr: compute03(active, since 4w), standbys: compute01, compute027 u& i+ R/ i. n* K6 J3 s$ i6 Q+ T- D
osd: 6 osds: 6 up (since 12d), 6 in (since 3w)0 s, @% B; } s4 E! r0 O
rgw: 3 daemons active (compute01, compute02, compute03)
3 W; `/ n1 d) n. k9 a ]
# { a" { s9 \% f* A: @. H task status:% m0 p) v3 }& a- @) g( C* t
$ E1 e5 H) T5 S# Y2 D5 g- W- x
data:
, q, Z+ f$ o+ u4 t) l: ^4 y/ K" [ pools: 10 pools, 1568 pgs
2 U( V. v H0 c' q objects: 65.20k objects, 258 GiB
& R7 Y+ u; M& C usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail, s# h* L/ p+ }3 n: E# U7 ]( r( S
pgs: 1568 active+clean: q* I' w3 h- D& I4 E. K5 p5 e, D
$ m O- Y& q! Z9 v
io:
3 v) x% h" n: W' ]( `* T" y client: 0 B/s rd, 15 MiB/s wr, 10 op/s rd, 51 op/s wr: }1 Q) Z# t. k1 p' }
! f7 R: I1 Y5 {' b5 B3 O1 \4 D
' {) b0 F& r+ e+ `5 o
, i! z! f! e# W9 ^1 a
b5 z3 z }) b* z( f1 J4. 首次访问服务
+ ~ t- s: T9 R/ m' e复制
" f5 A' [4 _; X7 `* u' r[root@compute01 ~]# curl http://compute01:9000/
. T' `) w6 a6 `1 R<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>
% X& K3 B: A+ b/ j, l8 f5 K[root@compute01 ~]#
! A l7 s$ ]2 [+ k
; A6 S# X9 b7 L$ n" H t提示错误，使用了 anonymous 匿名用户请求所有列表，到这里说明安装完毕了;( a$ P- |3 D* c' h9 C* T
; Q# q+ Z# }4 L0 Y* t: E
5. 修改 RGW 的 默认端口 7480 到 9000( q" }0 X% ^* L9 w2 g0 X6 Z
复制
+ U: {! K8 Y; P[root@ceph-node01 ceph-deploy]# vim ceph.conf 4 i! K4 C+ E) l' a/ n7 Z
。。。 5 ~. k! |% I8 W/ p1 J3 z
0 G+ I: @6 e Y[client.rgw.compute01]
* r; [; r# g3 O6 ohost = compute01
7 A6 Y* a6 Z" H% t( mkeyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring
' l3 b# f& x: o7 Q$ G1 P9 `) u8 Srgw frontends = "civetweb port=9000"
K& T. z6 `1 @. ergw print continue = false3 \* B; @9 H5 C7 m5 s$ a6 L+ I
: D* ^7 v g- U2 n! Q6 C* U
[client.rgw.compute02]# r7 G4 H% u. ^6 _* }
host = compute02
* r- D& b! _6 s. n% a, wrgw frontends = "civetweb port=9000". z* F6 C; U u( U0 E/ l* B1 L1 c
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring
$ y: M5 l7 j/ O& t# t/ B5 Prgw print continue = false
+ H; s# U5 Q0 M6 \7 U6 |- n: B8 R% M4 }
[client.rgw.compute03]) j" `2 A( v- k- P: }
host = compute03/ e' x# V; @ u* m c: \6 {0 O
rgw frontends = "civetweb port=9000"
. z$ b1 F% {. H3 mkeyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring: `) ?% w4 [4 y' y* b
rgw print continue = false6 S, o$ n! U- \8 O
! e+ X0 Y0 D6 R5 _
- O$ Q/ d$ a3 U修改配置文件ceph.conf，为什么修改这个文件呢?因为后面添加节点时，默认是copy的这个配置文件，修改这个配置文件，可以确保集群的唯一性，下面推送配置文件到所有节点;- \+ L, ?2 B! ~2 p$ f
* F' H. {7 x( I: P2 x2 J# k复制
1 a Y- a% X( H[root@ceph-node01 ceph-deploy]# ceph-deploy --overwrite-conf config push compute01 compute02 compute03
: ^* |* T& p7 d1 B/ @" V9 v; @& b6 m* k* }7 j: W
--overwrite-conf 注意需要使用这个选项，否则提示无法覆盖;
. ^' ^' w$ b" ]( |
+ b' {6 l5 J5 K, c: @% x- t6. 重启服务
5 r, `2 u% z0 v! f# V, K5 b复制; u( W5 u. {' {3 B6 @
[root@compute01 ~]# systemctl restart ceph-radosgw@rgw.compute01.service
4 E' H H& L* A h[root@compute01 ~]# netstat -antp |grep 9000|grep radosgw
' r) m" Q' G3 e4 s* R2 gtcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 2758818/radosgw 8 E" C8 G: F0 }! x9 H
[root@compute01 ~]# 4 [7 [* ]$ Y' A0 i" P# Q7 ^3 h
7. 验证 80 端口
4 g) ~7 A- i6 V) Q$ O4 \4 y! H复制
2 t- I2 O+ w0 P2 b. w[root@ceph-node01 ceph-deploy]# curl http://ceph-node01/ ( [" U' e* H2 T# T: K. p' C/ c
<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>[root@ceph-node01 ceph-deploy]#
; W" Z5 A3 ]5 s* p. P P M% m! A8 \7 f$ s Q
8 k& \) }7 u% I' a直此，RGW 服务部署完成;
% X) Q3 P$ w5 ]/ y/ C* N
4 g( n2 X: W2 a2 @, ]0 W使用 S3 访问 RGW/ J- T7 Z3 k, U. R% O: R$ i
1. 创建 s3 的兼容用户
1 L/ ^. I0 a8 \( n# b0 C复制+ X# s9 @! Z: l" |
[root@ceph-node01 ceph-deploy]# radosgw-admin user create --uid ceph-s3-user --display-name "Ceph S3 User Demo"
; l6 k1 o8 i+ _ n: d* b{ # v y. U# f" l: Z9 U {
"user_id": "ceph-s3-user", * N7 a2 F6 O& h7 ]3 L# b+ J. V
"display_name": "Ceph S3 User Demo", ( I+ |. R/ M. y# I
"email": "", 2 }* q3 }+ ?# @9 b; V) V6 C
"suspended": 0, / N: a, e X* Z) U% M0 h9 l. ?" Z
"max_buckets": 1000,
5 |8 }# _7 m. f& l Y* v, v "subusers": [],
7 [# J2 I# d4 v, G6 h, o& v5 K "keys": [
( {7 x, ^ r+ w) o( \ { 6 |- s9 J6 P6 t6 J
"user": "ceph-s3-user",
2 I" L( H/ @% @' ], G) s. |6 r "access_key": "V3J9L4M1WKV5O5ECAKPU",
4 P: }5 [) a' N, e/ A. c "secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw" , o3 N+ b( ~( g+ O G8 J2 r
}
( ?' e2 Q2 c9 M4 Z# O6 }) Z" I ], 2 E1 I3 r) n; C* L0 Q% X- A
"swift_keys": [], 2 V, A4 [) G, x0 ~- B
"caps": [],
. s o7 s+ F4 I8 A "op_mask": "read, write, delete",
' U$ Z0 Y5 |2 T4 Y* Q4 ] "default_placement": "", ' h2 u+ A7 O6 q( D ~+ H
"default_storage_class": "",
0 y% _/ ?1 F% N) g# B$ G: n' a "placement_tags": [],
7 O5 c! Q% }) \/ K4 b+ e "bucket_quota": { ; r! Q) D1 `* y# ^
"enabled": false,
: R& a! Q* Z5 ^ "check_on_raw": false, . X* Y6 w0 h h
"max_size": -1, 9 R+ C! w) ?( ?6 z' s
"max_size_kb": 0, 7 `2 E1 Y* W+ N; n8 a
"max_objects": -1
7 L! z9 g2 I1 t% ] u+ P! @ },
' q: } T$ P+ ?$ u "user_quota": {
; Q9 S* o) i! S; v3 g. E3 [1 q "enabled": false, + F% Y8 A7 O2 O
"check_on_raw": false, ) g& p( h- ~* i& x# j1 o- \
"max_size": -1, + ?9 g3 z# K1 B( _- X/ X7 ^
"max_size_kb": 0,
/ u! i! B/ q+ u8 F5 `( C "max_objects": -1
+ r) {" E, d* V7 D6 J },
7 q/ X0 z6 R& p3 x% g- B# i R% v) m "temp_url_keys": [],
9 e6 Y& q' n U2 Z- a( w9 s8 v "type": "rgw", % y2 n( \% t7 {
"mfa_ids": []
# \3 o0 `6 \$ U! T! K} ; ?& ^4 n; n% E- W. N
3 a4 a" c4 d% E# [% g: i% R[root@ceph-node01 ceph-deploy]# u( J) l; L7 t8 h1 c' x
/ F$ W# {* D# ], ~/ }+ ?# e! M* D* R: W5 O8 w- T
注意上面的 access_key 与 secret_key 很重要，要记下来，以备后用; 不记录也没有关系，我们可以使用以下命令查看;5 S) J1 F( T6 N: g, o
: F2 R9 X" G0 D3 o0 J
复制' j; ^0 ^( O) E
[root@ceph-node01 ceph-deploy]# radosgw-admin user info --uid ceph-s3-user , T5 O8 W: y: M( v
1.7 b U0 I. ]: \+ B8 |( W
2. 使用 Ceph SDK 访问 Ceph 集群
0 O! G4 t0 l- u官方SDK 使用说明：https://docs.ceph.com/en/latest/ ... g-s3-api-extensions' |) C) q( D/ X+ ^ J$ {
$ N7 F: [) m1 B4 i复制
$ m% p' ^ w: V+ k+ C# h# O% o; j[root@compute01 ~]# cat s3client.py g, r# H2 u6 ^ W( E6 \
import boto # h @0 b5 A0 r5 K; u
import boto.s3.connection + _ t" y* I* c: J
access_key = '8Y28X33O2YJKI3H6B1WJ'
6 S, i* T H$ f2 Q5 ~' L+ R. b3 Asecret_key = 'ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y'
/ m# n) z% ~. M5 O( G0 w2 s. g
4 \4 B0 P$ e0 Q6 [8 L3 S# g/ M, \conn = boto.connect_s3(
1 u+ b( N [9 f4 j aws_access_key_id = access_key, , f: I6 b0 i; X. b+ e
aws_secret_access_key = secret_key, ( Q7 h5 j/ h7 {. }. Q: |& l$ i4 w5 o7 K
host = 'compute01', port = 9000, * w" e, O$ @: I0 N0 W
is_secure=False, # uncomment if you are not using ssl
9 v( y: ?/ K4 G6 q8 k0 b calling_format = boto.s3.connection.OrdinaryCallingFormat(),
" I9 ]9 h/ ~# @3 I6 q ? ) ' ~/ y }+ Y% s* z5 A
bucket = conn.create_bucket("ceph-s3-bucket") " h9 Q0 p: ^8 q" { _) A
for bucket in conn.get_all_buckets():
. B* t7 K" D Z% z! i8 l print "{name}\t{created}".format(
7 T7 m; s5 }9 E9 s. Z name = bucket.name, 5 O; T; Z* \1 \6 ], e
created = bucket.creation_date, : A4 D# n6 c6 U( G! v$ ^6 d
)
8 l1 w" B$ ?% ^; a. R0 V& \7 ?0 X3 N
[root@compute01 ~]# python s3client.py N5 ^! |8 f- F! Z. J
ceph-s3-bucket 2022-02-09T06:35:13.546Z5 r- e# [/ y1 ~
安装完rgw后，会自动创建3个pool，一个是rgw.control、rgw.meta、rgw.log，当我们创建bucket后，还会创建一个rgw.buckets.index pool池;
: t& Y# J2 b+ c$ n5 l/ A5 l$ \* L6 N% }
/ X3 k6 r# |# V6 t7 h0 |复制8 d/ v2 a6 j b/ C; R' C: r0 X
[root@compute01 ~]# ceph osd lspools
1 `8 m9 k5 O2 C" e/ r5 images
w! b/ A5 e D- i6 volumes+ w9 n8 r) e( }5 a( X
7 backups; W8 Q: z; [6 I8 L- o
8 vms
5 i+ V7 V7 Q5 X$ `- t/ Y9 ceph-test
1 r% @' `7 w5 P$ q! q) y/ E10 .rgw.root
9 D! E& E' M* X11 default.rgw.control
; T; G8 V1 }* R12 default.rgw.meta* |( q4 ]; ?& i: ?* l$ t( g1 q' I
13 default.rgw.log
/ d) W* X L4 x% ^2 T( F2 C' Z14 default.rgw.buckets.index
) A) l1 p7 u, W1 S6 N- ?) A) l3 E. R! _" m7 l0 t) ~
& Z# a- q+ I E
3. 使用命令行方式操作 rgw
( K8 f9 V: z$ y1 g安装命令行工具
3 d" Y4 {: ~- u, X
% y& n2 o; ]0 S I1 @) O1 F复制9 J$ Y) N5 B5 t' p
[root@compute01 ~]# yum install s3cmd -y
6 }: C; s+ z/ p1.
: y3 ~" c# X7 O5 E( {* c' X9 N8 d& _配置命令行工具5 r* P, O, e. }
1 r' Y3 C8 M. @: j
[root@compute01 ~]# s3cmd --configure 3 f3 f/ U$ F% w1 }; n1 E' `3 t
9 s2 x5 A% W9 {+ z! C. K5 m: A
Enter new values or accept defaults in brackets with Enter.4 @# f2 A5 Y/ |, V" r# G
Refer to user manual for detailed description of all options." V, D' U r; {. Z5 z5 u( u! n
1 A5 f$ W$ }% |3 ]" n- Q
Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.9 n. w. C9 O2 X. U
Access Key: 8Y28X33O2YJKI3H6B1WJ; G& T% F4 s) `; l3 W0 e
Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y- b) ~2 n( O& K" \/ y
Default Region [US]: . T' ^5 j) ?% t9 W
- Z @- y- G1 F L! k# {5 h0 GUse "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.- F, M- ^7 o' e- m. r0 K& V
S3 Endpoint [s3.amazonaws.com]: 192.168.0.75:9000
/ M1 q6 K% t/ ~! g# K* Y+ a+ S) a. \2 _. x
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used
; K/ {5 u0 a: i- Eif the target S3 system supports dns based buckets., @: d! p' w4 ~0 W) N4 R" H
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: 192.168.0.75:9000/%(bucket)s0 y6 W1 a I1 y. N7 y) q4 y
2 s q7 l5 [* ~/ D0 b! ?5 i4 ~Encryption password is used to protect your files from reading2 ?* k/ }/ s# u- p- R. X
by unauthorized persons while in transfer to S3- g# s* r% D0 ^# G
Encryption password:
8 t* y+ z$ g' A9 \6 Y4 j3 jPath to GPG program [/usr/bin/gpg]:
$ i4 F+ q1 M B$ P7 V6 W9 |5 ^/ ]7 h& C; ^: z
When using secure HTTPS protocol all communication with Amazon S3
& l0 m- c' w: Q. B8 p/ wservers is protected from 3rd party eavesdropping. This method is# ^$ w7 B" P( p$ i' E: ?) l
slower than plain HTTP, and can only be proxied with Python 2.7 or newer
& a: n1 J3 h5 `0 dUse HTTPS protocol [Yes]: no6 V7 E ~0 E1 J* G, q0 Z# m
( S; N3 Z- t% [9 E2 o1 L4 Z2 N
On some networks all internet access must go through a HTTP proxy.
# F/ E; e; D. T$ ATry setting it here if you can't connect to S3 directly
2 i2 A6 P& f- THTTP Proxy server name: 9 h% D; J2 J% \, Z) C' N
) b; [7 M* ~" r' m- ANew settings:
8 z4 j4 V7 ^. S6 m$ G. v Access Key: 8Y28X33O2YJKI3H6B1WJ
R+ |, i% r9 _* ? Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y
5 b! D4 i7 O7 n, k Default Region: US
7 C: ]5 W: \: [3 ^% A S3 Endpoint: 192.168.0.75:90003 b" X+ @. H' S! k
DNS-style bucket+hostname:port template for accessing a bucket: 192.168.0.75:9000/%(bucket)s
# H0 D3 s* @+ }- G4 j2 o" l" r Encryption password: ! T5 [# l4 n$ \/ g ^( s* b& [% _! o
Path to GPG program: /usr/bin/gpg2 D& m, A" Z& c
Use HTTPS protocol: False
- V; z {9 p% M1 Z! d( r HTTP Proxy server name:
1 k: |& m" U! y* k6 c' F) z HTTP Proxy server port: 09 ^8 U4 W! L( s
8 `2 y$ I* z) B6 W% N
Test access with supplied credentials? [Y/n] y
, X6 v7 c: }3 H9 r/ q m9 NPlease wait, attempting to list all buckets...
- R* T6 B% F: q5 p pSuccess. Your access key and secret key worked fine :-)
: `* a# z8 i* u8 |% e3 T( e( k9 e. r7 i
Now verifying that encryption works...
8 i" ^4 [1 R9 \* ] b- F3 @3 j6 {Not configured. Never mind.
/ y0 s7 w8 e6 Z+ }- M6 U, c" ]/ K) E7 }" N3 E/ h) n2 }
Save settings? [y/N] y5 ~! }. M( {+ q+ {, V4 X
Configuration saved to '/root/.s3cfg'
: e9 o; r- N* B. j6 i4 `; V
" F9 g: N- j3 A
6 _* R4 ^% c. r1 n) C U) C% E! {! c; q
简单使用命令行工具3 s. s: _8 z8 R/ X' p
4 h$ E; ^$ z3 F/ D7 r5 \0 _9 P[root@compute01 ~]# s3cmd ls
/ A4 Y3 t: K$ K9 x7 r3 \2022-02-09 06:35 s3://ceph-s3-bucket8 z& {5 ]# `: ~+ I) R
. r1 m. G8 k% m
[root@ceph-node01 ~]# s3cmd mb s3://s3cmd-demo
' y) A8 _# m1 H" v3 J( zERROR: S3 error: 403 (SignatureDoesNotMatch)
; j4 x: @( U! H, \# Q[root@compute01 ~]# s3cmd mb s3://s3cmd-demo
6 @( U4 g. t5 P# n" [ERROR: S3 error: 403 (SignatureDoesNotMatch)6 e$ N; Q& w& k
! w; b/ l3 Z4 `* l
.这是需要修改版本，启用v2版本即可
2 t# N8 Q2 s6 ]8 G' m+ ]( @* D Z5 @. O: c; g
复制
% v1 {8 a5 C7 Y8 i& {' a, N V[root@compute01 ~]# sed -i '/signature_v2/s/False/True/g' /root/.s3cfg ) Y. p/ A* x+ Q" ]& z$ |
/ i9 t$ R) S a- s8 I5 l. p
0 I& H( u* o3 J3 |
再次创建$ h: s- p" b/ h( ^
& E4 S+ M4 P: M' z0 X
[root@compute01 ~]# s3cmd mb s3://s3cmd-demo
; K7 [4 m' i S! z7 ?Bucket 's3://s3cmd-demo/' created
1 ^( |6 S6 r- f# J4 h" w$ r6 q& y
0 S6 }$ L. ~/ D7 b1 \. o
' C8 w: V. x |) w, D1 x; Q3 X5 C上传单个文件
$ h9 p7 T8 X6 z0 q6 D6 D" ?* ?/ Z% E+ b! i2 u; r) Q
+ U- n5 E$ R* S$ f' g: G1 {& Q
[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo . N2 k8 E9 Q: p6 r) s# ~* t2 n
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
7 Z/ ~# t2 U. M4 {; d( A' h+ D) A, e 465 of 465 100% in 0s 67.71 KB/s8 [4 y# A% |0 d# a- c3 |
465 of 465 100% in 0s 67.71 KB/s failed: q' J' ~ K0 z5 C1 b, w' ~5 B
WARNING: Upload failed: /fatab-demo (timed out)
! P) @4 w7 i6 e: q7 R$ nWARNING: Waiting 3 sec...' q9 { j2 ]- `- d, E; S
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
# ~9 Y2 G; e" K 465 of 465 100% in 0s 361.83 KB/s+ D8 P w$ a& Z s' j- D" K8 `
, |! q3 P( A, x5 {% Y* F
出现错误 ERROR: S3 error: 416 (InvalidRange)，原因是上传object对象的时候，需要创建 pool 存储数据，创建 pool 是需要 pg，当 pg 数量不够的情况下，可以将现有的 pg 数量调小，或者修改配置文件调整参数，然后重启 mon 进程，调整方法有三种：+ K z" I. {2 Q' B$ B! l# X
" s6 u9 i' A! Z( o
调整 pg_num 和 pgp_num ，默认参数值均为8;7 n$ q$ y) \! ]- g
调整 mon_max_pg_per_osd 参数，默认是 300，适当增大;当每个 OSD 中的 PG数量超过这个参数值时，就会报错;https://www.suse.com/support/kb/doc/?id=0000194026 A3 e8 i- `- ~9 _( \) G/ L
增加更多的 OSD 进来;
8 K5 h. [3 \2 X我们采用第2种方法：
2 d ~ w' ]4 r9 o7 D* O( d* G! O+ L
[root@compute01 ~]# cat /etc/ceph/ceph.conf $ c& v8 Q8 v4 _) ?4 M. ]% Q' g& r
[global]
3 j- G i$ `6 f. r2 ?fsid = a6d0359e-8295-4e62-8620-54273462e79f
6 m4 x; ]8 S2 S( Y* amon_initial_members = compute01,compute02,compute03
$ t' d. g U, a7 x+ G9 l" t e0 Hmon_host = 192.168.0.75,192.168.0.77,192.168.0.79
5 z2 c: j# V; z+ e" a' Iauth_cluster_required = cephx
+ L6 \6 W+ p1 y6 a) oauth_service_required = cephx- \6 ^* a4 n" u! S6 z* @* X* u! v9 v
auth_client_required = cephx! t& b9 F' C; d8 D+ j) \1 Q U
public_network = 192.168.0.0/24 - }0 s# l+ r: W6 I) i
cluster_network = 10.10.10.0/24
+ y* [/ S; W. H: ?1 J \rbd_default_features = 3% k/ ]5 A& ?8 g9 f- H. H2 p
mon_clock_drift_allowed = 2* J4 P: f2 N( S2 h! G" c
mon_clock_drift_warn_backoff = 30
s2 j" k- B, \ _osd_pool_default_size = 3
% }( K" g) c' p$ V% w1 ]osd_pool_default_min_size = 2. H5 Y! U/ Q% b0 j/ V& ]& k7 E
mon_allow_pool_delete = true
U" x& ]" w) Y1 e' gmon_max_pg_per_osd = 1000
t% F% l/ d: E' t: V( s/ Xosd_crush_update_on_start = false
Z( m1 s D6 c1 y u" tmon_pg_warn_max_object_skew = 50) @; h& F8 G2 h& r5 B. C
" S2 @9 l* \2 T& l
[osd]" R/ t' k$ @; O+ H3 [/ i
osd_heartbeat_grace = 20
: A% w! q: G$ s0 V: hosd_heartbeat_interval = 5
9 ]" P9 U3 @7 p7 d, e- g8 s6 W# h0 f/ c* Y8 h/ ^$ S* |$ ?7 Q
[client.rgw.compute01]
. y* m% b$ p! ~ Q( _) Ghost = compute01* u: @2 l7 J5 v# ~1 ?: [# l
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring
4 ?: b; ~" W) ]2 g5 `9 c* @( mrgw frontends = "civetweb port=9000"
! G& G( `* {( b* Z( qrgw print continue = false
4 ?( L+ r4 B, M4 ] h* w7 ^3 P- L' C" W* }$ @
[client.rgw.compute02]# W t# w- w& a, w2 ^, K7 \3 U
host = compute02
5 a5 n: F& }" S* b6 k/ U2 d, ], L! ]$ Srgw frontends = "civetweb port=9000"6 \$ V5 X9 d4 Q/ ^: J Y
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring4 j; v2 U+ s5 b5 x+ y) j. d' a
rgw print continue = false1 b* _( v) H. _1 L
3 Z* W- Z- v0 f0 w[client.rgw.compute03]* x* i# z$ }6 X c1 W; ]8 M6 b X
host = compute03) b. T$ i+ @4 D7 v' P" x; l
rgw frontends = "civetweb port=9000"
0 E' ?7 s0 I7 S1 g+ h* Jkeyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring$ u( n/ G2 t* S' n& _& `: e/ Y4 J
rgw print continue = false8 P/ T" `9 b- t4 F5 y
% K. z& p- N' a
重启下 monitor daemon 进程, k! ^7 x F% ~% B7 k- q
" M4 `5 Q8 \) b( G6 v; A+ p
复制
; {1 m& _2 U% |[root@compute01 ~]# systemctl restart ceph-mon@compute01.service
8 K( B( `+ I& _[root@compute02 ~]# systemctl restart ceph-mon@compute02.service
+ }5 y& Q0 z. L& f[root@compute03 radosgw]# systemctl restart ceph-mon@compute03.service ; W' C9 X9 a4 t9 x
, X4 q% x. T5 Q1 N0 x
. H- n/ g k5 P- V, \, x) ]再次上传文件测试0 ^- |' g6 r. h( E0 h
/ W6 w' [" u$ ^3 a6 D6 j* A( J
[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/8 A& S( X0 b: P* [. b0 b Q( s" W
upload: '/etc/fstab' -> 's3://s3cmd-demo/fstab' [1 of 1]3 @' T' y$ K$ i9 k% C! q1 C5 m
465 of 465 100% in 0s 66.26 KB/s9 b6 T- R) D# f, P) v6 L
: J, J" r) d/ G5 t一直卡住，应该是哪里有问题：
9 ?; P3 r- H" x0 Z3 G5 M检查过程：1 O; p. E$ D( C t7 C" M0 i: z* r
[root@compute02 ~]# ceph -s
: y: d, y$ E: q, Y8 t" b9 `( J cluster:2 Z3 ` d' j# N' [. ~4 G# Z
id: a6d0359e-8295-4e62-8620-54273462e79f
0 h' U% }7 G7 y5 K8 b2 K- O) O health: HEALTH_WARN$ N' B) i% J: B9 Y1 t6 g3 Q4 ]
Reduced data availability: 32 pgs inactive1 j# J" m0 [/ T1 t7 O
; u$ Z t! a ` j! ] services:/ G- K/ R# {, G: z, q/ c
mon: 3 daemons, quorum compute01,compute02,compute03 (age 3m)6 D8 C! q& B( }' I0 L" ~
mgr: compute03(active, since 4w), standbys: compute01, compute02
4 h% s' g6 C7 \ d+ ~% m osd: 6 osds: 6 up (since 12d), 6 in (since 3w)
! {0 a+ Q8 v1 Z1 L! P5 f rgw: 3 daemons active (compute01, compute02, compute03)2 C( ^/ P: m( _ D
3 Q, n2 i$ D. e task status: h5 V- d" C+ a( ^- m
3 B; g; U: Y& ]' Q
data:% M. F7 g+ ^+ n# O6 z- D
pools: 11 pools, 1600 pgs
' f+ S9 f! M7 h objects: 65.22k objects, 258 GiB
) g) z" _, B3 e usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail
. z- G/ O) z) Q" L pgs: 2.000% pgs unknown h0 d5 J" Z. A& A
1568 active+clean
, t; B3 g4 s( ]2 ~6 z8 S$ N( }- { 32 unknown
& E$ G6 Q6 Z" G1 y, `% D" H- L+ q/ H! L* P, f1 \+ W% ~; C
io:
" J/ k# t$ m7 ? ~9 A client: 114 KiB/s rd, 9.3 MiB/s wr, 144 op/s rd, 53 op/s wr
4 j! q1 Q- [# _$ @5 m! E0 Z- O. D! ?2 h+ M) G, F. Y6 ^
[root@compute02 ~]# ceph osd pool ls C" x* I- V- [, }& z
images* a- m7 s/ i5 }) c* U4 J2 @
volumes
. ]3 I" l+ K5 Ybackups$ |/ ]3 K" k- J P$ b! U) Y+ D
vms
' R Z: v$ U2 m- S- [+ E5 eceph-test
' K/ c! d6 q! \.rgw.root
( Z( L9 K; V O5 w m' h9 jdefault.rgw.control. ?2 x% ^4 Y! r" F# ~
default.rgw.meta
& p2 w+ F; W6 a1 S; d9 jdefault.rgw.log* ^9 J8 o8 I# v0 k7 d+ c
default.rgw.buckets.index1 {2 f$ i' p+ V: t" t
default.rgw.buckets.data4 \7 y( v& ?/ H. Z, G/ X* t
[root@compute02 ~]# ceph osd pool set default.rgw.buckets.data crush_rule ssd_rule# o0 m" H- W+ R/ P2 B5 C
set pool 15 crush_rule to ssd_rule9 o( U, B; V, D7 c) }- m' S
& W$ f& b8 \1 U: B, R& P
' D1 q# K0 V" e$ S# i$ N* E完成之后，直接显示成功了： 如下done
) ?+ K P! {: g9 T
8 N, q5 b; z F% [7 e, Q4 R& D[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo9 m- R) ]8 q+ p! Y6 @! N8 r
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
# f- J& R* I6 Q: r0 G: u" K5 X' A& Z. U 465 of 465 100% in 0s 71.34 KB/s: v1 l) K0 v4 r5 O. k
8 o4 F$ r& G3 ~! F+ L" z+ t# T: f1 x* J. y. u k
465 of 465 100% in 81s 5.72 B/s done0 U0 _ q: x/ g9 [2 R, N4 S
4. 常用操作; M3 M+ e$ r: A0 H1 o
复制
% ]- x$ k8 E1 }5 S# 1. 列出所有 bucket 8 J* J) c/ N3 ?
[root@compute01 ~]# s3cmd ls
. N4 [# |. i$ r6 U2022-02-09 06:35 s3://ceph-s3-bucket
7 f* |( c8 g, o$ P' N' P" ]2022-02-09 07:45 s3://s3cmd-demo$ g/ v% J3 l! m1 ^1 m
7 M# z' G; t Q& {
# 2. 创建 bucket % W+ a7 P+ ~1 y7 Z
[root@compute01 ~]# s3cmd mb s3://gwj-demo/( z1 W* F& ~: o: Z
Bucket 's3://gwj-demo/' created1 c# R' I( ^! d. q- K
4 O# d9 y( I! G, m
[root@compute01 ~]# s3cmd ls
7 D. V4 t4 [. g* F6 q' j% j6 a3 M2022-02-09 06:35 s3://ceph-s3-bucket
9 O# C: [$ m3 j6 e+ g4 D2022-02-09 08:01 s3://gwj-demo
) l q8 J0 u) G: a" F. h, _% K1 j2022-02-09 07:45 s3://s3cmd-demo: d% A/ ]6 r" Z+ R
J7 L0 p- P2 |
8 Z9 d: \1 _3 q2 I# 3. 删除空的bucket : G) M' k& I' m8 m4 M' J% R$ v
[root@compute01 ~]# s3cmd rb s3://gwj-demo
( G$ [: z) ]& `$ @9 x0 JBucket 's3://gwj-demo/' removed
& Z) q# E, d" }
' `/ Q) T d$ S' k7 F7 P) ]4 ^! _[root@compute01 ~]# s3cmd ls
, P( V: M4 g3 M/ s2022-02-09 06:35 s3://ceph-s3-bucket+ T7 Q: Q. y4 e3 S; a
2022-02-09 07:45 s3://s3cmd-demo
. o2 G& z, e' p. a$ H2 j [# I7 l* z: r
# 4. 上传文件到bucket ; W" Z4 B6 E2 h; ?6 Q
[root@compute01 ~]# s3cmd put ip s3://s3cmd-demo" Z- c u" B* e8 N$ P" J% [
upload: 'ip' -> 's3://s3cmd-demo/ip' [1 of 1]
! w5 S0 w9 I2 ~" e* {+ ^4 R 5 of 5 100% in 0s 296.75 B/s done1 P# @7 o# w! { i' i
* l4 F+ M" L2 x. C( g& F
# 5. 上传目录到 bucket
% X6 W& k% j+ [7 F h0 t7 r[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/1 d8 e1 q/ c# H2 _" F3 f
ERROR: Parameter problem: Use --recursive to upload a directory: ./
" I* I: A( L0 [' n% B; M% Z$ _( W- I3 n. ]) j2 G$ d, n
[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/ --recursive
& }% z' Y" [4 V7 x5 |- B: bupload: './.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' [1 of 177]& w8 u j6 a0 I7 ?, T8 o
106754 of 106754 100% in 0s 2.29 MB/s done% z# w) u" t- n# [
upload: './.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' [2 of 177]
9 u7 h. L$ S, ^& W. Y0 v# P( y 111789 of 111789 100% in 0s 2.13 MB/s done
* \/ n5 \4 t$ J7 [upload: './.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' [3 of 177]$ p% Y, N1 p7 l/ P4 f; U9 A* U
111789 of 111789 100% in 0s 1485.51 KB/s done
5 P+ V: G% r4 T; z
6 s2 C K/ B9 B2 N! [6 K" K; w
2 z$ C9 ?: }# l, K' d+ x$ g又卡住了：
9 C$ g! M5 o2 f _1 i! n* f# ^* Z检查：% H4 L' h6 g& Z- v! V
[root@compute01 ~]# ceph -s
( x$ t9 G. U3 j4 L+ M cluster:2 y2 L- M% N- l! D Z
id: a6d0359e-8295-4e62-8620-54273462e79f: ^& M2 J' |; ~; c8 E, }
health: HEALTH_WARN" K' G# _0 m4 I
Reduced data availability: 32 pgs inactive
( b0 h/ w7 D9 D' I+ e3 S" L) d' w, t e& q! d% E2 U2 ~7 c9 f- g
services:
' R8 _5 d( N3 q* z/ S5 f5 h1 X4 x mon: 3 daemons, quorum compute01,compute02,compute03 (age 15m)
5 G5 p: a% ]' r; n) a$ W% } mgr: compute03(active, since 4w), standbys: compute01, compute02
% A, l- L$ h8 W osd: 6 osds: 6 up (since 12d), 6 in (since 3w)) S" ^' n! @8 p# `; b% f# v2 `
rgw: 3 daemons active (compute01, compute02, compute03)
' j2 G+ F, i: k7 ~6 ]; O
& L4 ]( V) H: _1 _ I% y9 r task status:
- f* h8 m+ }6 l" o7 E; ]& k" J0 U& T. I0 l8 r, |. U; E
data:
9 ^% N1 ~" V: R$ _) r3 Q5 m pools: 12 pools, 1632 pgs
9 J4 {4 s# m, ^% y4 C objects: 65.40k objects, 258 GiB2 T+ I8 k* I' b+ _0 p
usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail
5 L% M' Y2 B- P* k0 t6 h: Z pgs: 1.961% pgs unknown. j, u( G- A' w+ Q3 T
1600 active+clean' @( L% X5 o9 _. K4 H& T
32 unknown/ E0 ?( C! h1 _$ O" A; ~6 b" I
) K* J4 ^ O+ j, L$ b* I! N io:, m2 ?1 }2 ~- L" A
client: 117 KiB/s rd, 11 MiB/s wr, 145 op/s rd, 50 op/s wr! z$ p- s+ Q. r# D& w8 n# b
+ M0 b8 b K& L" r6 H3 k8 H[root@compute01 ~]# ceph osd pool ls
' G3 D' `+ l: Jimages( z2 b* e$ z/ G' q1 J
volumes
. Z$ ?8 a+ R5 t }( \4 @backups
, }- t3 T; i" j( P4 O; I' w5 T* i) Z5 Jvms4 h8 A; ]) I( b' H, J+ q# v0 ]4 U
ceph-test0 i5 m r6 R& K6 R+ l& F# l
.rgw.root6 h" T) G3 F# R$ o1 j( H# q5 [
default.rgw.control8 V+ |4 ~0 h' t/ {4 }* ]& }) ~1 p
default.rgw.meta
& r! r2 S+ G- p/ z* Xdefault.rgw.log
% G7 {( D' l* H$ ], y$ n$ Cdefault.rgw.buckets.index
x# _* u- U, O7 c* J0 Ydefault.rgw.buckets.data4 {9 m' `' D: c# G, r; ]# D
default.rgw.buckets.non-ec
! V* s0 C6 q6 Q: u[root@compute01 ~]# ceph osd pool set default.rgw.buckets.non-ec crush_rule ssd_rule% w/ i3 ]) z3 T6 J' E9 o" E
set pool 16 crush_rule to ssd_rule
2 ]6 q1 i$ D3 J( v
/ O" t# I" W' c" G4 n$ i9 N- C$ P0 S. E
( x% l- z( [1 s) v8 P3 @. B设置完成，上传完成：
9 l0 h2 B2 x- z0 }8 r! ]' [4 Zupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 1 of 6, 15MB] [174 of 177]; Z1 x6 q }* I
15728640 of 15728640 100% in 2s 5.38 MB/s done% h3 X" e6 u" F0 z3 ] r. Y, ^8 y6 m
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 2 of 6, 15MB] [174 of 177], L A" M7 V& c% {5 ]/ z8 J
15728640 of 15728640 100% in 0s 38.25 MB/s done
; B8 B' I# I: {1 v6 H; T9 {' S. Uupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 3 of 6, 15MB] [174 of 177]/ E- y+ t4 R" I. M
15728640 of 15728640 100% in 0s 22.08 MB/s done
; D) |/ F9 w% P9 D* B$ Z! C4 gupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 4 of 6, 15MB] [174 of 177]' | d* y1 p" @% O" Y
15728640 of 15728640 100% in 0s 15.15 MB/s done/ A R% G8 u+ f
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 5 of 6, 15MB] [174 of 177]
& c9 D f$ H: `, H1 W y 15728640 of 15728640 100% in 1s 11.45 MB/s done
% D' c) e" P9 s( B4 Supload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 6 of 6, 11MB] [174 of 177]
6 x$ R8 H' r2 f2 z 11599872 of 11599872 100% in 0s 21.31 MB/s done
2 a$ g( L' o# \% a( ?, o( E* h& Rupload: './monitor/telegraf_system.conf' -> 's3://s3cmd-demo/monitor/telegraf_system.conf' [175 of 177]) x% Y! i! j4 @6 z) C
4322 of 4322 100% in 0s 87.82 KB/s done; K/ X/ Y+ F9 i$ B; [. r
upload: './osds.map' -> 's3://s3cmd-demo/osds.map' [176 of 177]
# w4 D `! _9 a( y* W# f 5895 of 5895 100% in 0s 120.84 KB/s done
* T. t$ _$ b% z7 t2 I' Nupload: './s3client.py' -> 's3://s3cmd-demo/s3client.py' [177 of 177]
( Q4 ?3 ]: S2 a1 z ? J 659 of 659 100% in 0s 13.74 KB/s done" X# P& ?) i7 T
& L/ T; g1 L6 q/ b+ J& c
0 l( B ]( `+ ]( z, e0 s
6 E# n# o, h$ D& [. V& O
# 6. 列举 bucket 中的内容 % _. Z9 A+ J( ~8 ~4 O9 E
[root@compute01 ~]# s3cmd ls s3://s3cmd-demo/
/ L" k& `5 k6 G, _+ |' i DIR s3://s3cmd-demo/.ansible/; X/ ]7 U/ R4 i; K# p, \
DIR s3://s3cmd-demo/.cache/
* F4 c8 a" e7 k% u5 P( Y DIR s3://s3cmd-demo/.gnupg/3 b$ w* {1 {' g
DIR s3://s3cmd-demo/.ssh/
: ~' G( O1 S2 G DIR s3://s3cmd-demo/monitor// N9 m2 F9 d' \" v3 f
2022-02-09 08:08 25196 s3://s3cmd-demo/.bash_history# U; W2 m. ~# x1 z
2022-02-09 08:08 18 s3://s3cmd-demo/.bash_logout$ `& D9 h! A7 C* q5 J% M
2022-02-09 08:08 176 s3://s3cmd-demo/.bash_profile
& |% _* B z. P, C2022-02-09 08:08 176 s3://s3cmd-demo/.bashrc
$ S! r: J4 D2 ?6 e* {$ m2022-02-09 08:08 1077 s3://s3cmd-demo/.cephdeploy.conf
1 X3 F1 ~2 J0 b2022-02-09 08:08 100 s3://s3cmd-demo/.cshrc$ F5 u- ~3 C" p' L( C) }
2022-02-09 08:08 42 s3://s3cmd-demo/.lesshst. s5 C, }* m/ ]
2022-02-09 08:08 2248 s3://s3cmd-demo/.s3cfg
7 }5 c1 M) j1 N; l2022-02-09 08:08 129 s3://s3cmd-demo/.tcshrc* m/ Q6 ?9 V! o4 d" i5 {% h
2022-02-09 08:08 6490 s3://s3cmd-demo/.viminfo7 B. S( V8 B: l( y6 A" N4 W# H
2022-02-09 08:08 34064 s3://s3cmd-demo/11118 f i. O" Q7 g2 }% G- E4 l r- H
2022-02-09 08:08 1797 s3://s3cmd-demo/anaconda-ks.cfg2 R) }0 B) Y$ f6 y
2022-02-09 08:08 86604 s3://s3cmd-demo/ceph-grafana-dashboards-14.2.8-111.el7cp.noarch.rpm
$ G0 U, |9 O7 O4 H$ U2022-02-09 08:08 9625568 s3://s3cmd-demo/ceph-mgr-dashboard-14.2.8-111.el7cp.noarch.rpm
( e1 r8 H; [# l; ]1 u; {' e2022-02-09 07:52 465 s3://s3cmd-demo/fatab-demo
+ ~; n( I4 v8 d U6 W2022-02-09 07:58 465 s3://s3cmd-demo/fstab, o& K, x! {, m( S( B- H' D
2022-02-09 08:08 5 s3://s3cmd-demo/ip
- G! Q. F0 w3 d) h0 W" e2022-02-09 08:11 5895 s3://s3cmd-demo/osds.map8 ^6 o i9 ]4 b5 J$ T
2022-02-09 08:11 659 s3://s3cmd-demo/s3client.py
$ ?% |' g2 @+ X
0 `* p1 `9 n& q' i$ V, v: }6 U. }# 7. 下载单个文件 + [" d, O3 x1 X1 x* i2 g- ?& \# [
[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip6 u0 g9 p8 W* q6 X
download: 's3://s3cmd-demo/ip' -> './ip' [1 of 1]2 w9 y7 x6 }5 ]# _2 }
5 of 5 100% in 0s 396.17 B/s done
) E9 [2 D7 N. ^3 P2 U2 e1 E. ]/ b6 ]: J0 u( r; ^
[root@compute01 tmp]# ls6 u+ t0 V: E E& P5 \2 F+ M
ip4 C2 Y5 o- l, J1 m7 y6 I. I! L# H
0 e1 X: M$ N$ e2 A# 8. 删除 bucket 中的内容 9 a; D. ^- p8 {3 X; I4 ]. F4 t1 F
[root@compute01 tmp]# s3cmd del s3://s3cmd-demo/ip7 ~6 b* ]. j4 z* t
delete: 's3://s3cmd-demo/ip'
+ e# C0 |7 ~2 Y$ e _# f4 Y4 h; c- ?8 e- w
[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip9 t# [4 N( H: Z8 u# G
ERROR: Parameter problem: File ./ip already exists. Use either of --force / --continue / --skip-existing or give it a new name.
% t1 N* J; ^, n9 ?5 G, i. k[root@compute01 tmp]# ( ? W" U. u% T
4 n; B2 _2 G! s! Y/ a' w1 e# 9. 获取对应的bucket所占用的空间大小 - D% s% h7 M! T7 Q
[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/) ~3 r% }2 ?; Q
104M 178 objects s3://s3cmd-demo/& X6 F) s4 \, x* s7 E
[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/.ssh+ l: \) s- }, e' A' _' Z0 N
4K 4 objects s3://s3cmd-demo/.ssh
$ X. L7 n% L9 E& @" t/ N, v1 O0 R+ X
# 10. 查看bucket文件信息
; t% P( {( x" l+ P[root@compute01 tmp]# s3cmd info s3://s3cmd-demo/fstab8 h$ ?* a+ @: z. i4 p/ R1 U
s3://s3cmd-demo/fstab (object):3 s$ h8 j* E% Q6 K1 g. i" d% T0 V/ Q
File size: 4651 p- s# p5 G9 J' r
Last mod: Wed, 09 Feb 2022 07:58:45 GMT
S. }5 s. s M7 S2 I MIME type: text/plain
+ F( {8 i2 z8 F6 o Storage: STANDARD' R! M5 U: X, G; \! d+ `
MD5 sum: b3b65116db83a743d6e80ae6c31896ef
- G! C5 B6 _5 {! }7 G: x SSE: none7 Z) C$ j6 n8 `0 Y+ ^& k. M3 A
Policy: none
4 C) R" ~5 r5 s; a CORS: none3 f0 Y q0 I* Q0 M% ?
ACL: Ceph S3 User Demo: FULL_CONTROL" t) s% L; K' T5 X
x-amz-meta-s3cmd-attrs: atime:1644387423/ctime:1638501772/gid:0/gname:root/md5:b3b65116db83a743d6e80ae6c31896ef/mode:33188/mtime:1638501362/uid:0/uname:root
Q6 d+ r* ~7 z* M$ Q h2 [/ a1 q6 C+ ^3 x; n2 ?
# 11. 两个bucket之间相互cp % x0 w. n& i9 u
[root@compute01 gwj]# s3cmd cp s3://s3cmd-demo/ip s3://test-demo/ & }% L' v" m7 H2 O7 W4 i3 j
remote copy: 's3://s3cmd-demo/ip' -> 's3://test-demo/ip' 6 g, v9 y& j, n; g+ i( }1 a
[root@compute01 gwj]# s3cmd cp --recursive s3://s3cmd-demo/.ssh s3://test-demo/ . j% s5 |5 \' Q' ^) B
remote copy: 's3://s3cmd-demo/.ssh/authorized_keys' -> 's3://test-demo/.ssh/authorized_keys' 9 U$ g- }! l, ~4 O4 s& U: |
remote copy: 's3://s3cmd-demo/.ssh/id_rsa' -> 's3://test-demo/.ssh/id_rsa' . _4 l9 S2 k! ^% k
remote copy: 's3://s3cmd-demo/.ssh/id_rsa.pub' -> 's3://test-demo/.ssh/id_rsa.pub'
* m/ p. _# N% j8 w* Aremote copy: 's3://s3cmd-demo/.ssh/known_hosts' -> 's3://test-demo/.ssh/known_hosts'
% |4 e0 D- ] z4 D5 [/ q[root@compute01 gwj]#
5 k# q$ V3 _* e3 r: c# i+ p
* s6 I. w- V- N, g f6 u& [. q5 k# 12. 两个bucket之间进行mv操作 : \, H O0 `/ w
[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp
2 M$ F) j! r9 T9 y/ G5 w2020-10-06 10:24 12288 s3://s3cmd-demo/.swp
, ]% e' m! N& N+ |: Y2 H& t; W8 y[root@compute01 gwj]# s3cmd mv s3://s3cmd-demo/.swp s3://test-demo/ * b, i. P8 @" m8 p8 f" _0 G3 `/ o
move: 's3://s3cmd-demo/.swp' -> 's3://test-demo/.swp' ! A' [3 N a& r' u6 A/ H
[root@compute01 gwj]# s3cmd ls s3://test-demo/.swp
3 _" v8 {0 ]3 [, v9 X/ E3 F7 l ~2 u2020-10-06 10:36 12288 s3://test-demo/.swp
6 o) x7 _8 Q5 E9 i6 X[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp 1 `$ u* ]9 n# c0 j8 O! r4 q
[root@compute01 gwj]# 3 d3 P8 v$ o/ j r$ b' Z8 r6 u% p
" M/ D$ Q) I: [ V8 v* K
# 13. 列出需要同步的文件和目录，但不进行同步
( {7 W6 H3 ]( Q: L& l- S) f[root@compute01 ~]# s3cmd sync --dry-run ./ s3://s3cmd-demo 2 G$ V( ^* k4 |6 r5 y
upload: './.swp' -> 's3://s3cmd-demo/.swp' ! z# Q" u- ~# \6 h6 `7 {
upload: './ip' -> 's3://s3cmd-demo/ip'
& o* x- T1 f/ B! v+ \# x5 supload: './.cache/abrt/lastnotification' -> 's3://s3cmd-demo/.cache/abrt/lastnotification' , }8 d, h3 ?7 u5 }
remote copy: 'size.log' -> 'gwj/size.log'
% u; G+ n3 ~$ G( p5 E+ FWARNING: Exiting now because of --dry-run
]2 w/ Z$ ? j2 C[root@compute01 ~]# * u0 i: S8 W1 M9 `5 s6 ~4 s& Y2 S D
; R5 G& H! p+ S6 b5 V( F/ P6 Y; s, p5 V# 14. 在bucket中删除本地不存在的文件
) z& X6 u, s' [+ L5 v$ `[root@compute01 a]# ls
3 x8 e9 ]. J; X' a0 ~; B& W10.txt 1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt 9.txt 6 P" g2 t' d/ O. x% l/ K. {8 T5 a& ^
[root@ceph-node01 a]# 6 K0 {: O) Y2 r9 p2 A/ K e
[root@ceph-node01 a]# s3cmd ls s3://test2-demo/ 7 D' E5 q9 }$ E
! V8 o( Q; v; W[root@compute01 a]# rm -rf 10.txt
. W, ]( t# @9 C" v3 [* G( L[root@compute01 a]# s3cmd sync --delete-removed ./ s3://test2-demo/ , z. D) c# X; z/ c2 P, u
delete: 's3://test2-demo/10.txt'
0 p2 I0 P2 ^ P& b8 A9 W7 V[root@ceph-node01 a]# s3cmd ls s3://test2-demo/ ) p; X' C4 N( }3 Z0 d
4 C8 P1 b7 j% i& f! j; Q7 K
[root@compute01 a]#
9 }5 [) d+ e: B, Y& B1 Q, D! H; e3 ?1 k& z4 g" \
4 ]. A6 F3 l7 u使用 swift 访问 RGW+ ^+ x3 j: g5 q7 ]
1. 创建 swift 用户7 O! ^5 n4 \5 y5 x. o; H4 t
复制! J, c! L6 ]7 o5 r' ~2 n& \
[root@compute01 ceph-deploy]# radosgw-admin subuser create --uid ceph-s3-user --subuser=ceph-s3-user:swift --access=full ' z: g* p% q& H/ @ m! b# m2 w
{
( \; d2 E0 k1 f4 h* {7 I* y S "user_id": "ceph-s3-user",
5 W: J$ c. r' _ "display_name": "Ceph S3 User Demo",
4 N0 m* R! t, ~3 Y" t "email": "", * V" b* T: r& u a& ~
"suspended": 0,
; q8 A" S. {* j2 O4 w "max_buckets": 1000,
# F! E$ Z! U% N! q "subusers": [ & t2 _9 j+ R/ E: F
{
! j M! `0 ]# Z/ m "id": "ceph-s3-user:swift",
a" u( v \7 V/ p6 |- t "permissions": "full-control" ' {" U; H+ B/ d) r5 b3 q2 l" H
}
% e1 R9 o& |9 Z N. p: P3 [ ],
: W! K5 k4 l" V. c8 @8 s1 n% k% U$ s1 s "keys": [ ( L7 t/ t E5 [7 ?4 n0 x2 p6 a& `/ n* F
{
* i' y( h& u% y W' K "user": "ceph-s3-user", + y( u$ r3 J B0 p0 Y
"access_key": "V3J9L4M1WKV5O5ECAKPU", , j: T1 j* ~2 q/ ]- L& x0 l
"secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw" ) \& G+ ]8 T2 H8 X. @
}
: ^% F5 t4 G9 Z0 {; d7 l% c ], # ?6 v: H) d3 }& j7 S: v8 E$ O
"swift_keys": [
" l9 Y8 N$ ~) S% O9 K {
4 k: b8 K5 o4 V2 u8 q4 } "user": "ceph-s3-user:swift", ' g% y! A# H [& k2 Y, h6 n7 y5 D
"secret_key": "ZIOOU8Xcfe3m6ZZapK5P2rU0GGPaiS31chy9yvMW" " k7 q* U( @, u7 m9 Q |9 R! W
}
? E N5 G; t% X6 o- K ],
: B, Y4 J% [& h: {# D0 n0 n "caps": [],
: n$ _6 y" f" N5 _+ e "op_mask": "read, write, delete",
# D9 h& |" }' g, h# n "default_placement": "",
% G$ ~6 m W/ n( N& A8 ~ "default_storage_class": "", 3 j# Q S$ f0 M1 G
"placement_tags": [],
; O, C0 B% Q# b% ` "bucket_quota": { & z, t9 w3 k0 M2 S8 d. E
"enabled": false, ! r- N! W; V4 x) y1 f
"check_on_raw": false, 0 ?/ w6 n! l9 |$ Y" y4 y
"max_size": -1,
; T9 j9 F. t9 W# }9 Q "max_size_kb": 0, 7 W- N0 r4 O- {" x3 `* S
"max_objects": -1
- v2 D7 j! M- g. z* g; ~+ u },
* m2 N+ b) Q& @. ?' V0 | "user_quota": { - j" |/ T. I$ `+ c6 Q
"enabled": false,
" q' ^+ w) `7 j! X! g. [+ w "check_on_raw": false,
2 z2 x/ I$ l6 y "max_size": -1,
" f; J0 G! n2 ^) ^ "max_size_kb": 0, 8 Z% U: Z( l. V0 |& H3 k
"max_objects": -1
) Y: c1 E. M- Q; R: P9 ? }, . Y; y9 O# P. C D8 R' y
"temp_url_keys": [], 9 Z, b' a/ Y. u j1 U2 k
"type": "rgw", - q1 g2 O- V1 M" _; q1 J6 \
"mfa_ids": [] - x; W# }9 G/ Q# f
}
9 ~% J$ u; V+ R" E2 G: }! s1 K7 l, Q8 H0 U' _. e" P7 G
[root@compute01 ceph-deploy]#
6 n& K' x8 J0 U* `2 N, B; F! G; B: n: p. S/ M" @% A& ]9 o
0 a& {/ B" |# d- N: @7 |- e2. 创建 swift 用户的secret* J, P$ b( D A* }5 U. g8 B. r# p% ~3 S
复制
! \, n: |+ u: U% z! P* s[root@compute01 ceph-deploy]# radosgw-admin key create --subuser=ceph-s3-user:swift --key-type=swift --gen-secret
+ o- ?; ]2 O( z' X+ f7 P8 g+ b$ d$ i{ # d1 W0 X: }! I& f1 X0 B
"user_id": "ceph-s3-user", + N) j7 \7 o3 B& U3 E* B/ N! v/ [* @
"display_name": "Ceph S3 User Demo", 8 c5 ~' A8 G5 Q( r* h- P
"email": "", * t. u- H! {2 Z( ~ m# }
"suspended": 0,
6 z7 L |/ c u- i, q& {3 y# q "max_buckets": 1000, 3 q9 l+ d6 G) y) y8 l' K" d
"subusers": [
$ _9 C( G% j& }/ E& d5 m7 O8 O) H4 u& ~ { 5 ?+ ~! F9 t4 }" g C9 \
"id": "ceph-s3-user:swift",
0 Q4 q) [4 E* x2 Y$ w+ |- K "permissions": "full-control"
. b% I# `3 G& U: N8 t( y: V& o }
Y7 y3 r. h6 C/ W6 ?0 {6 J ],
( N2 x Y0 F, a9 {+ e# u' M! x "keys": [ # t* P) ]; g# y% m. L6 ?9 E( t5 z
{ # R2 G( K' r8 c4 `: I% J) Q- \( ?* z
"user": "ceph-s3-user",
9 w9 H7 ^* k) W# O+ u "access_key": "V3J9L4M1WKV5O5ECAKPU",
6 h H# K" b: C& | "secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw"
/ e) b( ?* Z; W; t- l } ( I6 b: M/ C- K" @* u5 g
], 6 d1 ~) c9 a; n( A7 x
"swift_keys": [ . q0 c( |* P) }; ]- {
{ 2 V+ k- \4 S. c7 Y8 G4 \6 ]
"user": "ceph-s3-user:swift", 6 v0 T- ^! ]4 L. ~9 I9 S9 z
"secret_key": "0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE" " P g9 c; i- b* }. ]
}
1 p( V5 h3 a0 i ],
: Y9 l8 g, k2 t& @ "caps": [], Z6 `1 i, t! Y; e% D
"op_mask": "read, write, delete", 8 Y0 j5 X& G' X7 `/ M% r& M* R4 T
"default_placement": "", - I' T( O! T0 d1 l! I: q
"default_storage_class": "", 6 `$ }# W/ ?( L5 X; M" I$ N
"placement_tags": [], i) h5 t ?- I" P; n8 y6 U0 w
"bucket_quota": { 9 c; S1 V0 I8 J
"enabled": false,
( ^( S+ d% E! W( [6 T4 G "check_on_raw": false,
1 n" }7 L4 l! y O "max_size": -1,
) L I- V3 u$ u "max_size_kb": 0,
$ N. v7 p2 J. b" D "max_objects": -1
0 F( p& r; K0 i x" |9 C }, & Z& {/ ^( m+ ]/ k- ~1 D
"user_quota": {
% O* `# k( P7 T; x( P "enabled": false,
! r4 T0 D5 K% j6 ~: k6 o "check_on_raw": false,
( {$ y; _* e9 C6 J% ^; r "max_size": -1,
Y( N2 }7 t# ^6 ]% D5 J "max_size_kb": 0, 7 w4 Z) }/ ?2 F3 U$ x3 w6 l# h7 D2 D F
"max_objects": -1
( b5 u0 g( Q1 V1 @4 u5 k5 z }, 1 Y4 `7 k. N0 F: Q# w6 g
"temp_url_keys": [], % J( H5 G) J# K6 u, K
"type": "rgw", 9 A* J& v: o8 ^4 p) @7 g- \: z
"mfa_ids": [] / D; q% S6 h: `1 b- Q
}
0 X R6 Q7 E' u6 c" }3 b
- n' T- Z. o- K/ R& h1 i[root@ceph-node01 ceph-deploy]#
1 ^8 |5 B3 T" m/ A0 r: v7 Y4 F/ S7 V: G
, G6 ~3 A- g* P
3. 需要使用 pip 安装 swift 客户端工具- h9 J z% O6 C7 e% G
复制
9 q+ f2 q, Z& F# 注意如果有pip的话，就不需要再安装了 . b# X) V9 r+ F
[root@compute01 ceph-deploy]# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
/ I6 B E V( v7 L* Z: m" J; P。。。 ( k* ^. a" H/ Z* M8 L8 Z
[root@compute01 ceph-deploy]# python get-pip.py
* N: o2 c# x5 Z4 ?! X2 Y。。。
& Q% S) [! |7 K x ^[root@compute01 ceph-deploy]# pip install python-swiftclient
5 T6 y- y5 u& j) L8 V, X6 u。。。
6 X( W7 i" {3 v) e/ g9 o
# u+ n. z# p* z Q* x$ f: ^
0 ]5 |5 Q' \! e4. 使用 swift 命令行工具. O+ N+ ^; j( l5 ~: F
复制
% y. T- L* |) M) N5 o0 F[root@compute01 ceph-deploy]# swift -A http://100.73.18.152/auth -U ceph-s3-user:swift -K 0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE list
9 R1 l* R$ g8 r, R' M( cceph-s3-bucket & o: p6 l, J7 L( I
s3cmd-demo
2 H% U2 ^" L3 j; w3 R[root@compute01 ceph-deploy]# & R7 j' k& ]7 v- Z' w
2 d7 C& J. W0 ?) T d* ~% h7 F( _: V" e5 y
. `2 b* X: z W5. 配置成环境变量的形式
6 Y) e- V( N5 K7 O7 H+ V3 g复制7 R L% ^' E( y; O) v1 \
[root@compute01 ceph-deploy]# cat /etc/profile 2 f# I* G- E/ t4 R% R7 d0 c1 G
。。。
8 I0 z/ o9 T4 F, t, B/ l; Hexport ST_AUTH=http://100.73.18.152/auth # J0 K. X) E7 m6 [
export ST_USER=ceph-s3-user:swift
+ S: Z. q2 Y+ _6 @+ i2 X/ Cexport ST_KEY=0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE ! i8 n+ e6 o5 `' h# K/ p' m
[root@ceph-node01 ceph-deploy]# source /etc/profile 2 C/ f8 Q/ |1 M C: v
[root@ceph-node01 ceph-deploy]# swift list ) Q- ^* ~9 j7 d( I' s
ceph-s3-bucket
, w& J5 j$ v1 W; Vs3cmd-demo ! o, @& k' k d7 {6 V3 G& T) M
[root@compute01 ceph-deploy]# 5 O( k1 ~: S5 J/ M3 @) o. `
/ T: _6 @ x7 f3 {2 i
0 i' o& c8 `" a- m% k+ _( k6. 创建bucket
! A! X4 N0 A& G8 C I9 j, E复制
u# G7 w9 [: g) T u( W[root@compute01 ceph-deploy]# swift post swift-demo
5 p2 j& C9 I! Y, W/ x[root@compute01 ceph-deploy]# swift list 6 T# o5 ^* G" @4 p/ ?! c% k* w
ceph-s3-bucket : t2 V @! v4 ^5 k* Z2 K; ~! \7 x
s3cmd-demo
8 g; d: T; J& Gswift-demo
: ~! J" w4 m# F, l# \1 D[root@compute01 ceph-deploy]#
$ k' I/ O; @' x I8 |6 Z1 J5 z: K, H- h
2 u$ _* i) t2 T" y9 R) o$ j$ w
7. 上传单个文件测试
+ O( v: \6 C% n0 e W2 O复制
) r/ _/ r; n* M0 q5 H/ G[root@compute01 ceph-deploy]# swift upload swift-demo /etc/fstab 1 U k2 _& ]# q* ^* T) v5 p$ c
Object HEAD failed: http://100.73.18.152/swift/v1/swift-demo/etc/fstab 416 Requested Range Not Satisfiable " _; o- s" Q! u. ^. I
[root@compute01 ceph-deploy]#
3 M t# C+ o$ D/ V" x8 D$ n/ F; H7 V! t
8. 上传目录测试
+ d: ]; r: n8 S4 a复制- i& L7 B1 a" @/ ?& }* f
[root@compute01 a]# swift upload swift-demo /etc/fstab / P+ u6 p: x6 k; Q. p- f2 b
etc/fstab
1 n1 ]$ \. V4 y, f: q( W[root@compute01 a]# O; o1 ~8 y o! z' R& u7 l
' |7 v2 S( J3 ]
9. 常用操作
! f0 l2 |4 [, l0 r6 Y复制
2 r- M' }7 Z/ h! N& j* T9 \; I# 1. 列举所有 bucket
N+ A5 I. L. {# ?5 d& X6 E4 K6 B[root@compute01 a]# swift list
' y6 `/ |( C2 o; W5 k) Oceph-s3-bucket
1 C1 u( k6 A- q2 C! J, Ws3cmd-demo
" N6 M) Y" p* Z1 i- z- T# j4 Lswift-demo
1 K, j% _9 T) _. M; t! i. ]test-demo
; ]) i5 m6 [( l2 v7 ytest2-demo
+ T! V1 t: O8 Q4 T+ n7 P( j[root@compute01 a]#
, k8 M0 [" b' S* w) |: n( Y& a' Y0 `8 z; M* }/ k3 D
# 2. 列举所有 bucket
: o) E+ l7 x: \$ `- G% ?$ }[root@compute01 a]# swift list --lh
`4 }* v2 z/ G; z* n 0 0 2020-10-06 04:13:10 ceph-s3-bucket
+ I# a3 r& @( A; b9 U! \ 37 3.6M 2020-10-06 04:34:49 s3cmd-demo ' w1 ?3 `# |' x- ]
2360 33M 2020-10-06 08:07:55 swift-demo
: y3 Z( o& A9 e! {0 e 7 16K 2020-10-06 10:32:02 test-demo
) W8 \5 F; E f- _4 | 9 387 2020-10-06 11:17:00 test2-demo
6 l4 W8 [% f$ @! E 2.4K 36M ! A; K) s! b% q' K$ V% T* ?
[root@compute01 a]# 2 U# M4 ]. S' B+ t a; h
& e: |* m; i* F, j8 A* j" @; H. m6 u# 3. 列举单个 bucket
( J( E4 _9 r* X6 W" S[root@compute01 a]# swift list swift-demo + @7 a6 N% P. l7 p& P6 z5 l
) Y2 T) Y7 E) `# 4. 上传单个文件到bucket
) j8 @' h, \' ]6 ~. }" @[root@compute01 a]# swift upload swift-demo /etc/fstab + X' G5 B9 K+ i- J) Z) H8 s f
etc/fstab
7 ]8 ~6 D8 ]- c! F- Z$ f% k[root@compute01 a]# 6 m7 |0 U. F W' N0 k
/ v+ y) n% S Y! _, s5 |
# 5. 上传目录到指定的bucket
& t* r; \4 w9 D5 _3 @+ A- O[root@compute01 a]# swift upload swift-demo /etc/
9 {* R' H2 Z% l7 X" m/ I
5 [; o* }" r, f" I0 w# 6. swift 状态信息
% h, n3 H7 _5 Q- \9 d. m4 B* {[root@compute01 a]# swift stat
}2 o, Z* ~5 S, B Account: v1
& B) N% J% S+ J. U( r! e Containers: 5 / B A2 {# n, \) L# Z3 e
Objects: 2413 - M1 \, [2 o* I) Q; B; m- E
Bytes: 38701415
1 Q' @) f8 _' H- LObjects in policy "default-placement-bytes": 0
3 i$ u. c& X. M& L8 k Bytes in policy "default-placement-bytes": 0
" U( H ]$ b* f2 u9 l Containers in policy "default-placement": 5 ) j s- r# O- O. D! f
Objects in policy "default-placement": 2413 8 P3 [% w/ C( M; k, x3 D5 a1 M
Bytes in policy "default-placement": 38701415 5 S$ L/ N: m; u
X-Openstack-Request-Id: tx000000000000000001302-005f7c5afd-a638-default ) i6 }6 @6 e0 `: d' P
X-Account-Bytes-Used-Actual: 45948928 % I/ q: K$ v: ^# W6 v: P! U
X-Trans-Id: tx000000000000000001302-005f7c5afd-a638-default 2 J; W! ^" ~4 n, o! c5 v$ a3 S
X-Timestamp: 1601985277.38095
0 A+ r: w3 ?- D: v* D Content-Type: text/plain; charset=utf-8
5 q/ c* O( S1 W M Accept-Ranges: bytes
5 T+ I3 E/ @+ y, H[root@compute01 a]#
" x/ ]. R# Z1 G$ Z; Z& G) K. ^
. w3 z1 t- q. b" T& R3 U# 7. 创建 bucket
% V6 y D( @4 \9 p9 d[root@compute01 a]# swift post swift-test b0 h2 {+ ^" a; p
[root@compute01 a]# swift list
7 a1 L) _7 Y( A* L7 o& Dceph-s3-bucket
" v p: |8 r: h; Ws3cmd-demo
- a( f& h( m* oswift-demo $ Q4 V$ g" p- A. `: M4 C% P
swift-test
$ U7 d) J1 H |; |# Vtest-demo
! c+ @, ~, g. v, Gtest2-demo 4 |+ b$ D9 `+ e4 E' c: s) Y
[root@compute01 a]# " n9 Q$ G8 I% v- Q# j' ^
% D0 s( h! `) ~/ \( J3 m" x4 n# 8. 删除 bucket + X3 A3 v. x8 B
[root@compute01 a]# swift delete swift-demo ! o# v3 T- I* H% b( k' R! @ E ^
3 J/ f, h- \9 C# 9. 删除指定 object
* k- _8 s L4 W0 t: N& V# h; z[root@compute01 a]# swift delete swift-test root/a/1.txt
4 n4 N) u5 V9 N3 Lroot/a/1.txt $ @5 v* R; u R0 O
[root@compute01 a]#
5 N' r5 D/ k$ Y7 j0 T) j1 r: i# T$ f" V
# 10. 上传大文件时可以使用-S指定分片大小 # X0 |- J: {; |3 R
[root@compute01 a]# swift upload swift-test /home/log.txt # b. Q) R' s% a, B1 I' Z& D
home/log.txt
6 p* j" S* [# X" {[root@compute01 a]# swift upload swift-test -S 102400000 /home/log2.txt / b# j: W" C8 w# }, p
home/log2.txt segment 5 0 s; X3 ?: S3 Y1 B" k! Q
home/log2.txt segment 3
1 b: [. M; ^ j9 R7 i) \5 shome/log2.txt segment 1
$ H# N6 i! u7 K# ^; Mhome/log2.txt segment 0 ; g6 I1 O% N& V/ ^' U
home/log2.txt segment 2
6 n3 g' Z7 J8 P' vhome/log2.txt segment 4
9 D2 J: N$ O% w( |8 khome/log2.txt 8 A) g; y! B4 y
[root@ceph-node01 a]#
1 F+ m0 R4 A9 w4 z; B
' _4 ]0 c$ E- }; U8 J总结
* C! U% \0 A6 D% \ y从概念、安装、到实践使用，简单介绍了对象存储命令行工具。
9 ~7 u4 K3 x! p3 ?3 q7 s7 U k
2 X* i1 N$ o8 W, M9 G- M r: E- l* H8 u( B2 s3 y8 W1 ^; X
" ~3 p# H: U/ k2 s* g0 ?! x( g2 Q! ^
|
|
发表于 2022-2-9 15:15:20