|
|
Ceph 对象网关主要是由 radosgw 接受用户请求，然后与后端librados交互;它主要起到承上启下的功能，对外它能兼容两种接口，一种是S3，另一种是Openstack Swift 接口，这两种接口，都有各自的用户认证机制，所以 Ceph 也提供了一套独立的用户管理机制，这套用户管理机制能同时兼容S3和Swift接口，最终数据会落在 OSD 上面，无论使用 S3 还是 Swift ，它们落到 OSD 上面，都是相同的名称空间;所以你使用 S3 存储的，也可以使用 Swift 进行访问。这是对象存储的一个基本架构，所以如果要使用它，我们需要部署radosgw，才能访问到集群，默认是没有安装的。- O8 v: b; q; O6 E- o: E
' x* l. C7 x1 j; Y
bucket 是什么?可以理解为装载对象的容器，它的后端是无限可扩展的存储空间，并且具备安全可靠性。Ceph 对象存储它后端是借助 ceph rados 实现数据容灾的机制，那么他能提供哪些功能呢?
6 }$ E o; ~4 z: H1 ^4 r* O
1 }+ c, x! _) p* q6 e基本功能
9 F' S7 h* ^/ W1 DRESTful Interface # RESTful风格的接口，实现上传下载及管理功能;
- A) L: s5 w' H. kS3- and Swift-compliant APIs # 提供两种风格的 API 接口，兼容 S3 和 Swift;% X$ f# w0 a5 [/ _
S3-style subdomains
( ^$ V6 y; Y; I9 c! _" O6 lUnified S3/Swift namespace # 扁平化、统一的的S3/Swift的名称空间;
/ v" K" Z8 w9 |( y7 ?User management # 为了安全性，也提供用户管理，可以限制对象是可以公共访问，还是授权访问;: N9 O7 \: |# Z& ?
Usage tracking # 追踪用户使用情况 rados df
6 z' |* q+ w$ p4 {8 V! m3 VStriped objects # 支持分片上传
; O2 D% [+ X: \8 T( JCloud solution integration # 支持云解决方案集成
S" z9 }# [+ U k$ v# F3 E' k+ k- N: MMulti-site deployment # 支持多站点部署4 ^& p( q; t) L9 V% ?( @# c% ?# h
Multi-site replication # 支持多站点复制
; f# o" s8 L m6 T, m安装 RGW2 i. t% W* v" M
1. 软件安装$ z$ z3 A/ n5 p z5 j3 t9 y
复制
0 v, l9 g/ @$ y[root@ceph-node01 ~]# rpm -qa |grep ceph
7 u1 D& c4 \- E4 C# [, ?' tceph-base-14.2.11-0.el7.x86_64
M" C) P# _2 \' _; x2 o8 l, Lceph-mon-14.2.11-0.el7.x86_64 3 R. I5 u5 Y$ y# I5 w$ X/ r: e
ceph-deploy-2.0.1-0.noarch
5 y% L( \* n# n1 I! [3 Dpython-ceph-argparse-14.2.11-0.el7.x86_64
" Q5 x) x. B* ?. D+ \. w* vlibcephfs2-14.2.11-0.el7.x86_64 n: R3 ^& T& |3 K8 t
ceph-common-14.2.11-0.el7.x86_64
, }. l) ^" h( m; ]$ Z8 Kceph-selinux-14.2.11-0.el7.x86_64
# _. m: b7 ]1 L+ \ceph-mds-14.2.11-0.el7.x86_64 7 m' m' e4 ^( p9 g. d I: o
ceph-14.2.11-0.el7.x86_64
( X, o% O, A" L, Z8 Cpython-cephfs-14.2.11-0.el7.x86_64 . p% B' s0 K, F6 d" N7 Y, _
ceph-osd-14.2.11-0.el7.x86_64
% M4 D: t7 R# S' X, U$ o! _ceph-mgr-14.2.11-0.el7.x86_64 9 Q$ @0 H% w5 |: I3 s
ceph-radosgw-14.2.11-0.el7.x86_64 # 直接使用 yum 安装即可 / D3 C- n/ y/ R+ P
[root@ceph-node01 ~]# 9 Z" F' r& D% ^' K9 M% O R
2 c) u# D4 T( b- H) Z* _1 [9 F$ y% C9 t% P
2. 启动服务，默认启动在7480端口
% C9 x2 s, ~9 o5 W* s7 X复制3 s% K% n- s) L4 {) p* H! `: |
[root@ compute01 ceph-deploy]# ceph-deploy rgw create compute01
3 f2 w% P$ n9 o% C
" P+ i6 r0 \4 v! D6 h4 O: t3. 服务检测% t; S* L/ K( p, D; J0 W2 K8 w
复制
8 L9 z8 t" l: e8 q[root@ compute01 ceph-deploy]# systemctl status ceph-radosgw@rgw.compute01
/ g. a9 z0 C! {- ~. K& Y● ceph-radosgw@rgw. compute01.service - Ceph rados gateway ( @( l# G F% Z" Y
Loaded: loaded (/usr/lib/systemd/system/ceph-radosgw@.service; enabled; vendor preset: disabled) 4 }& F W1 I) X: n' Z8 |5 i
Active: active (running) since 一 2020-10-05 20:34:36 EDT; 14s ago , t+ T# w- R9 Z0 |
Main PID: 33574 (radosgw)
6 y7 U6 g6 o, L( B' J- w+ | CGroup: /system.slice/system-ceph\x2dradosgw.slice/ceph-radosgw@rgw.compute01.service + d1 W# I/ b6 Y7 f" k' R; X
└─33574 /usr/bin/radosgw -f --cluster ceph --name client.rgw.ceph-node01 --setuser ceph --setgroup ceph
- K) @" |9 R, l0 d. |! u; m* x9 F6 ^
10月 05 20:34:36 compute01 systemd[1]: Started Ceph rados gateway. ) |9 F" e! N9 ?# z7 v9 T
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:13] Unknown lvalue 'LockPersonality' in section 'Service'
2 A2 K0 t1 r) q6 C8 j: O9 P$ j; S10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:14] Unknown lvalue 'MemoryDenyWriteExecute' in ...Service' + b- C/ B3 S4 w; t+ A
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:17] Unknown lvalue 'ProtectControlGroups' in se...Service'
2 P' F0 D9 V5 |# S1 f10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:19] Unknown lvalue 'ProtectKernelModules' in se...Service' 8 z- R# X2 w$ q, Q( |
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:20] Unknown lvalue 'ProtectKernelTunables' in s...Service' , B- F0 s$ }1 o$ D
Hint: Some lines were ellipsized, use -l to show in full.
% G, b: }- `0 g& |7 v& @/ n, C[root@ceph-node01 ceph-deploy]# netstat -antp |grep 7480 # t3 J! L: J+ R% \: c
tcp 0 0 0.0.0.0:7480 0.0.0.0:* LISTEN 33574/radosgw
7 r C* M& _! v[root@ceph-node01 ceph-deploy]# ceph -s . T+ E( v' j. f/ p$ }. }% U
cluster:
5 u; O8 M C, J6 z+ h! i$ l, Z' x id: a6d0359e-8295-4e62-8620-54273462e79f
/ Y5 Y2 W7 l( e7 e1 }+ b health: HEALTH_OK
( e2 r$ g' k, T+ x* z6 a/ U$ m; ?" ]8 u- I) Q6 p" ]
services:
# A7 q: G# n3 a3 y, e' q$ _6 E mon: 3 daemons, quorum compute01,compute02,compute03 (age 12d): B' `2 K( e8 v4 R/ c( _
mgr: compute03(active, since 4w), standbys: compute01, compute022 m7 i, Q: \9 F6 n* [! U8 ?+ a
osd: 6 osds: 6 up (since 12d), 6 in (since 3w)
4 s9 f( M% [7 e6 X. T+ h" B rgw: 3 daemons active (compute01, compute02, compute03)4 K3 P) k6 g0 u; `( Q( t
( D) `) c* _* n
task status:& A6 A& p2 ^: I( y% f( v5 c$ Z o
0 h% N0 t9 H. e8 V3 n6 K9 `4 X% R( u data:6 H$ a; e' r& |) _
pools: 10 pools, 1568 pgs
. v2 M+ f. w, R. c2 Z( g4 {( ~' { objects: 65.20k objects, 258 GiB
- H2 f( o9 L! V3 M* b s. d, A usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail7 O/ Z( P4 a/ b, }5 H# G" `
pgs: 1568 active+clean& J2 G, _2 D2 t# D: w9 t
0 V; W7 K6 T+ U" n3 [9 u
io:4 K* r, a0 |* f' [* s4 n9 t! G
client: 0 B/s rd, 15 MiB/s wr, 10 op/s rd, 51 op/s wr! e3 t" r& v: F: \
/ r9 C6 c8 m" ?( q3 b
7 {) Z; a+ C9 J$ o! h2 A. @6 H6 Y/ r, O% w1 Y1 z
- \3 ~4 I0 Z1 I( p/ E4. 首次访问服务
' J" b) @8 }- e8 q' F/ W& @复制9 U! H; J+ B% L
[root@compute01 ~]# curl http://compute01:9000/
2 s, V( I7 K* M: E& e1 k2 F. n% A$ Y<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>! E! g6 O' I( C9 W& Z
[root@compute01 ~]#
+ G* y7 u, N: K7 ]; @+ M/ N9 a, ~, }6 t y" }, p L
提示错误，使用了 anonymous 匿名用户请求所有列表，到这里说明安装完毕了;9 p! q& k$ \; {6 V, t1 R
( K4 w) K \# Z: d3 {
5. 修改 RGW 的 默认端口 7480 到 90007 }5 l* t, a9 \; O
复制
4 C% P/ B/ `; b2 N; e* w+ B[root@ceph-node01 ceph-deploy]# vim ceph.conf
% j1 z3 a( A! B7 B。。。
# v# h* R V0 o3 u3 ~# _' s+ H0 x9 f' m
[client.rgw.compute01]
% z( o1 q" l& C6 m3 |) ~host = compute01
$ n1 r8 D+ f1 _' t9 s7 e+ }+ t$ Y/ Vkeyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring
8 r, l- }2 d) `, F; ^; C5 Wrgw frontends = "civetweb port=9000"7 T0 Q. B6 N/ z$ ]) `
rgw print continue = false9 I0 G9 [, d2 u. P' s
5 K' t/ ~7 O# L% L
[client.rgw.compute02]2 t7 x4 \" J0 H" }( I+ ]
host = compute02
5 m, I! u* M' rrgw frontends = "civetweb port=9000"
+ P3 p" A4 B% s# n3 vkeyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring) d, d9 _& W' C& C. K! d% q
rgw print continue = false
1 p( u* _# }. {: v8 X
5 L8 B4 F, B. }% _6 H* b$ k[client.rgw.compute03]
9 f6 e. U) r( d+ Yhost = compute03' P% Y! o/ H5 X8 I3 V0 x; F3 G
rgw frontends = "civetweb port=9000"2 d' ]0 g! ~& {# U; \0 P8 J j% ^" M4 P* A
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring
& \) c: k: q. ]+ Srgw print continue = false
& g, E6 Q( a; O3 p1 t
7 F* T7 L% u/ l
7 Y! l) P `# c: o修改配置文件ceph.conf，为什么修改这个文件呢?因为后面添加节点时，默认是copy的这个配置文件，修改这个配置文件，可以确保集群的唯一性，下面推送配置文件到所有节点;
+ J' F6 k9 b3 o2 G' k! N% }
/ f1 r- H& D4 n复制
( U d8 ]2 r. b' \' b U! X3 h[root@ceph-node01 ceph-deploy]# ceph-deploy --overwrite-conf config push compute01 compute02 compute03
6 [- X+ w6 r& D q2 \
2 A1 S3 h- I, A. L6 ^* ]) ~1 r5 h--overwrite-conf 注意需要使用这个选项，否则提示无法覆盖;
( F0 t4 B, S6 @& {) F& V S o$ Y" m/ N/ W" k) o
6. 重启服务
* f8 d3 J6 p$ B5 |8 w& S+ ^复制* C' M1 K. Z; O9 p$ T8 y3 I( |
[root@compute01 ~]# systemctl restart ceph-radosgw@rgw.compute01.service5 W$ f0 T% J7 @$ J. _1 g
[root@compute01 ~]# netstat -antp |grep 9000|grep radosgw
# k2 [6 _2 v9 D- {% \# _tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 2758818/radosgw
) \' G- U/ X2 I( ?' z[root@compute01 ~]#
9 ]; s! N% z8 k! `5 U. c* f' m7. 验证 80 端口
" e3 [5 P* U9 `" k复制
; Z1 p" L3 h6 K0 [[root@ceph-node01 ceph-deploy]# curl http://ceph-node01/
5 L u0 n; o" `3 O# J v0 V<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>[root@ceph-node01 ceph-deploy]#
; D( F/ A5 p' n9 c/ ]5 ]
( |. K: B) H' t4 @, E0 R$ v( @/ n5 o3 d0 ~: P' i+ \( x$ b
直此，RGW 服务部署完成;: v8 V# Z4 {1 M4 e1 ^
( R. c# C7 h6 `7 @使用 S3 访问 RGW" ~5 S7 P. M1 `$ b
1. 创建 s3 的兼容用户, R! m m: M2 F9 q+ D, I
复制
' ~6 T' N: c/ i) P, ][root@ceph-node01 ceph-deploy]# radosgw-admin user create --uid ceph-s3-user --display-name "Ceph S3 User Demo"
3 G4 d- {$ ~3 `0 _% k4 ]{ ; @( Z( y8 a8 d
"user_id": "ceph-s3-user", ; e' w0 ?) x5 p- x0 a* M
"display_name": "Ceph S3 User Demo",
; Z# L: r& S: U6 l8 V: z, t+ e "email": "", 8 b5 f' H2 C( a% A6 \' i# o% A/ Y( P
"suspended": 0, $ ?) [$ {0 |2 ~7 T2 h
"max_buckets": 1000,
6 a" j7 C5 P: f! j" I* u "subusers": [],
* ^$ o4 B7 n' C5 [4 w "keys": [ # J9 w7 D* R5 [" ^8 B+ a3 K! t9 q
{
& x6 g) t7 h, l8 z# [, B "user": "ceph-s3-user",
* C0 m+ D& K9 b) I8 S2 h "access_key": "V3J9L4M1WKV5O5ECAKPU", 8 l3 R: [ c4 @6 s$ O8 V1 q2 n
"secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw"
% ]* s( i' r- Z! A9 G } , `5 O' O( V$ G" C
],
$ j$ k; B X4 D7 s0 F" v e# |/ v "swift_keys": [], 6 m: `* r9 [' A" e
"caps": [],
, F- e' E. `* n7 m8 ~" p/ J4 l "op_mask": "read, write, delete",
! M4 E+ w+ T+ P; r3 t7 ] "default_placement": "",
" [+ f# `# M5 S, B5 X5 m6 o" G2 W "default_storage_class": "", 0 J- |- o1 J9 g9 N P3 B0 d- r
"placement_tags": [], . c5 R6 `% q I" ?" T
"bucket_quota": { ; H- m d5 Q1 q4 B4 M% H6 z
"enabled": false, / ]2 x/ v# E2 g5 A8 ]" S! s
"check_on_raw": false, % f& l6 R4 p& b3 p. y
"max_size": -1,
* N2 y( o# x4 K& D* K "max_size_kb": 0,
- a3 x0 D3 d4 S: o "max_objects": -1 6 b+ d) ]! d' S( O# _
},
2 H4 C, N& a3 m# X) ` "user_quota": {
1 S4 o3 H4 m" j" `7 T( n "enabled": false, + w" N0 E5 R2 j% U. i4 v5 |
"check_on_raw": false,
$ _+ j. v+ q6 V6 [ S$ `) ? "max_size": -1, ! F4 O7 q+ }# P
"max_size_kb": 0,
: W; k9 @- d! e% H- b* x, U' F "max_objects": -1
- Y# P0 Z: n+ u+ a },
; ]3 p: Q2 i' R$ X# E "temp_url_keys": [],
+ J3 b2 j3 W9 D5 @" P "type": "rgw", " m- C: P# Z( J3 @2 U. f
"mfa_ids": []
9 N* b/ n! D* _' K' A$ ~} $ c0 F5 [! L1 k G9 X! Q
3 f# L- ]0 W1 A8 d6 r8 k l[root@ceph-node01 ceph-deploy]# 0 Q! U1 h( A+ x7 n1 \; \+ l
/ s+ S9 `1 A$ R Z, s: n
$ b: o: Q N% `6 ~# {$ J! I: a注意上面的 access_key 与 secret_key 很重要，要记下来，以备后用; 不记录也没有关系，我们可以使用以下命令查看;
& V4 ?0 u, X7 D) Q: C; m7 @1 ~1 f: B9 x
复制" J8 w% Z! G4 C! N# ?/ h
[root@ceph-node01 ceph-deploy]# radosgw-admin user info --uid ceph-s3-user
7 e3 F, ?2 V( J, [, W' U/ ~' n1.
! M$ h! c5 L5 Z7 v2. 使用 Ceph SDK 访问 Ceph 集群* U8 B2 @* \3 J4 l, F1 A/ [; @9 Z
官方SDK 使用说明：https://docs.ceph.com/en/latest/ ... g-s3-api-extensions
: T7 n9 t0 C1 Z
, T2 I. c$ K3 E5 G复制
: ^4 W" C& k: x2 t$ l. W. h[root@compute01 ~]# cat s3client.py ) V: _, x- k+ \. n& k& f: k/ ]
import boto
1 i# d0 x! m' }* A2 rimport boto.s3.connection 4 O# G" l3 B% N( P2 | k: `) q
access_key = '8Y28X33O2YJKI3H6B1WJ'
9 Y9 _" c3 T+ F' N8 _secret_key = 'ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y' / A# D# @. I. f, E- B
8 k7 ~( R5 x- \1 D$ {. Rconn = boto.connect_s3( 4 F9 l3 W5 X3 e
aws_access_key_id = access_key, 4 z# Z8 E4 I r8 K
aws_secret_access_key = secret_key, 8 F9 k0 R f+ |: C1 M6 Y7 e" W
host = 'compute01', port = 9000,
m! P1 P8 g" Q! t. ^; ~+ @. Y' e is_secure=False, # uncomment if you are not using ssl
- S- O) }6 u! q% r+ W% n calling_format = boto.s3.connection.OrdinaryCallingFormat(),
: c- g% o0 t6 D# J% |) q( [ )
# t. O6 O7 H8 s7 _) e$ Ubucket = conn.create_bucket("ceph-s3-bucket")
& a& y5 M) \. O7 h1 n* F$ Cfor bucket in conn.get_all_buckets():
. C8 A# L8 z! V! S1 E# k" r5 _2 U5 |/ K/ ~ print "{name}\t{created}".format( - d$ l4 q- K% \
name = bucket.name,
; R M! v# C9 k created = bucket.creation_date, 3 x8 n8 ]' _$ @+ l p% `
)
* n6 ?9 Q) W* Y# M3 ~
3 l8 c+ |* ?4 ~3 F[root@compute01 ~]# python s3client.py
4 a( \; w9 u6 e' G# @3 J) R+ D2 ?ceph-s3-bucket 2022-02-09T06:35:13.546Z6 S9 @. ?" R& D: O
安装完rgw后，会自动创建3个pool，一个是rgw.control、rgw.meta、rgw.log，当我们创建bucket后，还会创建一个rgw.buckets.index pool池;
& g* O |, f9 @
' c, M5 l6 g4 ]5 t: j5 ^* w复制8 S: ] w2 A+ O
[root@compute01 ~]# ceph osd lspools
- C. G s$ Y4 I* y. t5 V5 images9 n" f, O. s4 C& J, U, p
6 volumes* Z: D7 h( o0 G7 v
7 backups
2 M# k9 Z9 d- B7 ]- Q8 vms8 x. P1 C+ b9 ]
9 ceph-test! N5 R; `9 j+ p
10 .rgw.root; t4 x4 |% t; ]" ?9 Y# L; }8 x
11 default.rgw.control) l+ ?* l# n& _
12 default.rgw.meta1 Z5 ?4 p/ X6 n0 I( F. V# H* Q
13 default.rgw.log# y0 _8 q$ H5 u$ K' W6 j' e5 s
14 default.rgw.buckets.index* o) R! C6 M3 r3 c- O# {
' Z @. C: y' `4 U+ v' ?
; P6 r- c) z2 s6 j" o3. 使用命令行方式操作 rgw
5 }2 K4 M, l0 {安装命令行工具
/ u7 \/ ^: U: [. U/ G1 c* A7 @/ Q2 c
9 h& Q5 P! X* j7 ]复制
5 q' F+ a1 V9 Z+ A1 Y7 ~6 a[root@compute01 ~]# yum install s3cmd -y) {- u7 V4 D- G/ ^( x2 }8 T
1., |! \- U: M. G1 p! K
配置命令行工具
/ f) n; e' e5 ?% p+ h+ t
# o; H M# z# x9 J s; `& O; J[root@compute01 ~]# s3cmd --configure 9 A& a1 Z' c& [0 ]. J) s( j
9 F' N0 J% X4 w. ]8 K( H. K, \* J. Z
Enter new values or accept defaults in brackets with Enter.
8 Z( s% I! v6 X8 ^4 K# s' _Refer to user manual for detailed description of all options.& Q" z$ l+ m. C# X( a0 Q) n
3 m+ q6 j# x" \Access key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.+ W- \& r- |0 D3 C. [6 Q
Access Key: 8Y28X33O2YJKI3H6B1WJ
6 |8 T" z5 L: @! @Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y
* }+ N# |" G5 v QDefault Region [US]: ( H/ x) o$ Z! {; j% _) g& }' b
3 O# `9 M+ ]/ x* ]$ `0 A$ VUse "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3.( K$ g! }) }4 o" e& A5 e- Y5 Z
S3 Endpoint [s3.amazonaws.com]: 192.168.0.75:9000
9 \. c7 _. ^- c# I) e, F# E" O6 e' s- R1 h5 E, T# j! Z( C
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used" U% S, {( A8 C4 k2 H! U
if the target S3 system supports dns based buckets.% H+ W9 W# C7 } A
DNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: 192.168.0.75:9000/%(bucket)s2 D# i* b+ P% l* T# z& N# C4 _
8 C- L4 |, p: \' `) YEncryption password is used to protect your files from reading/ [; V3 _: O p( |; `, n
by unauthorized persons while in transfer to S3
) z. N. u, ? y! U, D, U4 ?1 `Encryption password:
* l+ a8 y2 q% o4 pPath to GPG program [/usr/bin/gpg]: * ]# L" u' x2 S5 |' x: A) S0 C
) T4 ~" A% B% }% m6 b
When using secure HTTPS protocol all communication with Amazon S3
; Z) \3 @6 q L5 `; bservers is protected from 3rd party eavesdropping. This method is
6 N' h/ ~0 N8 H' x9 G* T* Q( v9 vslower than plain HTTP, and can only be proxied with Python 2.7 or newer; g0 s9 e. l$ x, q
Use HTTPS protocol [Yes]: no2 p) d. S6 q. _4 y/ }$ r5 \# T
$ t2 {0 d1 T" y. c: x4 }
On some networks all internet access must go through a HTTP proxy.
: M( p0 D2 k9 }. GTry setting it here if you can't connect to S3 directly& J6 K1 O( ?& b" `
HTTP Proxy server name:
/ n4 d, t+ m# D0 |4 p
; O" z9 X: h. p8 N# y5 Q- `New settings:; l. I- Y/ k9 K, S
Access Key: 8Y28X33O2YJKI3H6B1WJ l* c9 t# b! y( f
Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y6 `" F8 h7 e4 q7 y0 e# }4 E
Default Region: US! r7 ^ F0 p& k; K
S3 Endpoint: 192.168.0.75:9000
8 ^- i$ Y. e1 k( w3 e8 D DNS-style bucket+hostname:port template for accessing a bucket: 192.168.0.75:9000/%(bucket)s % h1 _& y8 Q# I& P Y) \$ K3 X/ ^
Encryption password:
5 v$ ^# M2 C" C5 w8 H Path to GPG program: /usr/bin/gpg
1 f' V0 @/ S4 v Use HTTPS protocol: False, d1 }/ P- R. M. e L" }$ h
HTTP Proxy server name: 1 Z: g1 X& V6 {3 l/ m5 }' W
HTTP Proxy server port: 0
" e# L2 M4 @& h, Y
! H, ^8 ^" O, |7 ]Test access with supplied credentials? [Y/n] y
2 j, v% q5 c S1 q0 {" ]Please wait, attempting to list all buckets...
. T; D+ n" i/ b$ \- MSuccess. Your access key and secret key worked fine :-)8 M& y( w% j- @$ P) f w3 K0 H8 N( U9 x3 P
# J1 o: k; G$ O- nNow verifying that encryption works...* v/ S/ P, |; L/ F* X3 H2 Y1 J& D8 n
Not configured. Never mind.. I4 O1 Y" Q1 [
) {+ U) Y: J2 {( MSave settings? [y/N] y
, ~" K/ ?) X$ K8 g5 L3 [8 qConfiguration saved to '/root/.s3cfg'! U8 P& V5 L+ ^7 P" u" K8 s
* F3 A7 ~* ?( E
4 M9 e* U. S, v; R* z
1 A4 k" s; c- ^3 J. e' D# Q/ x
简单使用命令行工具0 U% ~& `( ]) B8 t* F, L
. @5 ~8 h6 g. x9 o5 ?9 L' }
[root@compute01 ~]# s3cmd ls
. }, Y) U }. ~! z6 u2022-02-09 06:35 s3://ceph-s3-bucket
5 s, b7 d. B3 k6 w, M; \3 k0 t% S, N" k% k: l
[root@ceph-node01 ~]# s3cmd mb s3://s3cmd-demo
: V; S: ^) Q: \- T6 x# ]4 Y/ bERROR: S3 error: 403 (SignatureDoesNotMatch)
7 B% ]6 D5 m9 a/ V7 u- d" Z; q; `' W; C[root@compute01 ~]# s3cmd mb s3://s3cmd-demo
: l3 k3 n+ Q: c! b( T2 Q/ S8 U) _ERROR: S3 error: 403 (SignatureDoesNotMatch); p, c3 a( ]" `3 k8 T+ R/ b8 B
( T% v& r+ b5 i1 r) G
.这是需要修改版本，启用v2版本即可
# }6 Y' i- h/ ]9 T* q( g; `1 W& i
* Q2 H) U0 ^! c# z复制
& M4 s0 M* k# B( \2 k9 y( J[root@compute01 ~]# sed -i '/signature_v2/s/False/True/g' /root/.s3cfg
8 i; y4 {( v/ d d- O9 G. Y; e7 Z. X/ ]. B- M
- M4 r0 h2 a( y% z7 k再次创建
) j3 E6 P! a# S& W3 {
) Q- B. i, z7 }0 y8 d, i[root@compute01 ~]# s3cmd mb s3://s3cmd-demo; u5 C: E2 J6 ]! ~5 z' b
Bucket 's3://s3cmd-demo/' created
2 }# u Z0 S. m/ U5 T" d8 e! E! ]% T8 r- |2 q
& L/ M2 V/ |5 {+ Z# i上传单个文件
- |& i& w$ T: }1 f; i) E/ L9 g$ B# J/ b* v6 N
1 m: K, _2 \: @: I% Q[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo * ^( g9 K7 E' R
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]% t, @% O6 H: e! p' G3 [6 q9 Y
465 of 465 100% in 0s 67.71 KB/s
3 A1 a$ q2 B7 |3 p 465 of 465 100% in 0s 67.71 KB/s failed/ {8 m. t1 Y! L% t; Z8 A6 g
WARNING: Upload failed: /fatab-demo (timed out)$ }! t; C) [, h. m \) Q2 g
WARNING: Waiting 3 sec...$ c8 K/ c5 S2 i \3 b0 S
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
0 R! g' u8 {; q8 S4 m 465 of 465 100% in 0s 361.83 KB/s
" `# J4 y0 }/ H4 _$ }* a4 h$ }8 S3 X# a9 s
出现错误 ERROR: S3 error: 416 (InvalidRange)，原因是上传object对象的时候，需要创建 pool 存储数据，创建 pool 是需要 pg，当 pg 数量不够的情况下，可以将现有的 pg 数量调小，或者修改配置文件调整参数，然后重启 mon 进程，调整方法有三种：0 S2 m3 Q7 m1 X6 G
& }3 B" m2 p3 F0 @
调整 pg_num 和 pgp_num ，默认参数值均为8;6 ?6 [" F; K4 V) ^& \" X. f
调整 mon_max_pg_per_osd 参数，默认是 300，适当增大;当每个 OSD 中的 PG数量超过这个参数值时，就会报错;https://www.suse.com/support/kb/doc/?id=000019402
% t! I) c$ \* @9 T: D& ~. f& \增加更多的 OSD 进来;
7 s3 o7 Z+ @6 v9 J我们采用第2种方法：
) C% W- X; Q& M- ?8 t
5 B, R. g: o6 a$ _[root@compute01 ~]# cat /etc/ceph/ceph.conf
/ y# z+ {9 ^2 {5 h1 Y2 ?" \3 C: m W/ M[global]
- l! x+ C) t! P- P; E( bfsid = a6d0359e-8295-4e62-8620-54273462e79f4 R( O% w# f! ^0 A
mon_initial_members = compute01,compute02,compute03) _# I) ?# x j5 ~
mon_host = 192.168.0.75,192.168.0.77,192.168.0.79+ K. i+ X- z+ f3 b. d% r
auth_cluster_required = cephx ^5 Y& K% Q/ w1 H
auth_service_required = cephx
5 s5 a& ?& [% s# f" r2 \auth_client_required = cephx I! g- M k+ w# p- M1 B! l
public_network = 192.168.0.0/24 ) i1 L9 {$ g; x9 O5 |, k
cluster_network = 10.10.10.0/24
9 ^7 I0 }( V2 l: Grbd_default_features = 34 f0 E( F# ^- |$ @1 M8 t
mon_clock_drift_allowed = 2$ C3 W# a2 y7 g/ B8 L+ I; [7 m+ R
mon_clock_drift_warn_backoff = 305 ^9 E/ x5 |4 \+ Y7 e
osd_pool_default_size = 3
/ E* w! m$ i5 V& f( X& F/ Dosd_pool_default_min_size = 2+ ~% \9 |$ l9 r' f1 W& Z( a3 U
mon_allow_pool_delete = true! X9 f2 V N9 j; }
mon_max_pg_per_osd = 1000
/ b6 s5 \6 t0 [% Hosd_crush_update_on_start = false
5 M1 H2 N% b( O9 N& U1 lmon_pg_warn_max_object_skew = 50
5 S+ t, `% R% A5 c" s' Q# ?2 \
% l3 V- I; D0 @ b[osd]
' |+ `/ x: {2 O/ h4 U" W. i( Fosd_heartbeat_grace = 201 [1 g& b; W1 t4 x; C
osd_heartbeat_interval = 5
: d9 k1 k% y& r0 n; f1 U* B5 ]' y" u* A/ d/ Z
[client.rgw.compute01]: e. L) o# G3 s
host = compute012 P$ O) _" i% b6 E6 r( J' |% H0 S# H
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring0 P7 P3 H* e! P& t' F2 O
rgw frontends = "civetweb port=9000"
4 n" C, u# q6 J" _rgw print continue = false
]$ O( u+ k/ T% a B' _! K. o( Q/ _* W. ?) R6 u
[client.rgw.compute02]4 c- z/ v8 z# ?5 X
host = compute02
8 E V. Q) e1 ?! W1 Srgw frontends = "civetweb port=9000"& F5 o) y3 @% T1 _- y; [
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring* C$ ]* o0 s. w2 Z b
rgw print continue = false7 `* Y2 Y; P" Z) \3 ~
9 R7 S4 o ]- J! T. r7 O9 r+ L[client.rgw.compute03]- V" ^1 ^$ a3 d# \: g, J
host = compute03; l1 ^7 S4 f3 j2 R; p3 T
rgw frontends = "civetweb port=9000"
/ K, L# X9 L+ _6 w! ?keyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring) j: K" | h+ y0 U
rgw print continue = false
9 G, R9 V N- \5 V- @) Q e& L7 M* J
重启下 monitor daemon 进程- @' w V3 M# Z- X: N. E
% |! ^/ e/ O: }! s) n
复制8 _- q: |* M3 M
[root@compute01 ~]# systemctl restart ceph-mon@compute01.service , Z1 T! l' }9 c; P: o) L9 `# }
[root@compute02 ~]# systemctl restart ceph-mon@compute02.service ! q, r3 q& p8 t0 Q
[root@compute03 radosgw]# systemctl restart ceph-mon@compute03.service ; j) l: a& G+ F, I5 y, j2 ]
' l, u6 H& H. N* d8 v
% {% {2 R/ m J再次上传文件测试
c/ B2 Q& {) u- F1 Z& A: h# D" {( z2 V8 _, h% @
[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/
( `6 ?# }* q4 \0 f* N, P3 N. |2 @upload: '/etc/fstab' -> 's3://s3cmd-demo/fstab' [1 of 1]3 ?# h; k6 b9 k; Y( o2 Y0 i. v
465 of 465 100% in 0s 66.26 KB/s; F+ V+ W7 U0 N/ W4 O
5 I7 }, r3 M, U( e2 A0 h; e. s7 i* J一直卡住，应该是哪里有问题：
6 H3 A" {/ U8 Q# i7 _% n3 `检查过程：% u2 m6 q d7 k9 @5 c- j3 L8 [
[root@compute02 ~]# ceph -s( B: X$ N4 k K, ~% R" E
cluster: J. L9 y5 x7 C3 f0 P- k; h
id: a6d0359e-8295-4e62-8620-54273462e79f/ G) b1 x' p2 S0 W
health: HEALTH_WARN! n) }) }7 C2 z8 s' m& A& Z9 I8 }8 O
Reduced data availability: 32 pgs inactive I/ I2 a2 G: c/ s6 f
0 V& i& C5 y$ T! o services:
+ J6 T7 @3 ?1 m- }2 w mon: 3 daemons, quorum compute01,compute02,compute03 (age 3m)3 t' [7 B( M( N
mgr: compute03(active, since 4w), standbys: compute01, compute02* S3 Y8 C Z2 c2 R$ j) `9 f7 B
osd: 6 osds: 6 up (since 12d), 6 in (since 3w)/ |* O8 {. X5 @
rgw: 3 daemons active (compute01, compute02, compute03)
' m3 [5 Q: m, F4 R1 g
! B4 W, C; c! _5 C7 W3 ^( y' B task status:; k, c ]# q6 G) c, S
$ F( O; r# d$ i
data:3 e# Q. M3 `* _& k3 F8 o% i
pools: 11 pools, 1600 pgs2 _+ ^2 }; ]9 G
objects: 65.22k objects, 258 GiB2 d" B. h B. L* q) A! v4 t+ `
usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail; ?: U. b" ^# M( t! d4 V+ I) ^% \
pgs: 2.000% pgs unknown c/ H7 q& ~1 l( I1 V
1568 active+clean
& B& r1 J$ B' S, N0 f 32 unknown) f! X0 T8 @9 B' o) ^6 e: B
, k- K3 D6 |% F3 L9 ~1 z! I
io:
" s3 W7 t* g4 E* s0 W0 C5 U client: 114 KiB/s rd, 9.3 MiB/s wr, 144 op/s rd, 53 op/s wr
1 c8 f! R4 S3 Q" a6 X' I) ^) }
" Y8 f0 K0 D( d8 }0 Z[root@compute02 ~]# ceph osd pool ls4 K* K0 B$ f' e& ]
images6 i9 o3 v$ f1 N6 T. }2 Y* c' n9 z
volumes% _) u5 u: i/ d/ I
backups
5 Z5 y/ w5 _; a% b- }; \7 U4 uvms
; a8 X1 a5 _& Y( C. Y1 i; dceph-test, Z9 T6 u: ?. @0 \/ x+ }: Z' U
.rgw.root) R$ p& c9 Y7 D' {' ?5 {
default.rgw.control
+ k1 v- H$ x4 O. O1 Pdefault.rgw.meta
2 f D" H# b" Wdefault.rgw.log" ? y# q: H* n' C+ c: Q3 K( i
default.rgw.buckets.index
8 V$ g3 |( x- a3 ^default.rgw.buckets.data
# V/ _' s" {# E2 c[root@compute02 ~]# ceph osd pool set default.rgw.buckets.data crush_rule ssd_rule5 a. f: r& @5 l; z6 K6 v2 b
set pool 15 crush_rule to ssd_rule
. H0 ^( p/ Q, B1 {& X" O
. R5 @/ C& j' |* T
$ x7 V. u: P9 X. C, x9 e完成之后，直接显示成功了： 如下done* Z( U8 v( E0 Q4 d# T. r# p. E) _
0 e. ~8 s1 K) t3 c) q* V7 Z" v[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo: w D6 K9 n1 _3 e0 H
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]* u( a$ M/ f% S* s
465 of 465 100% in 0s 71.34 KB/s1 `$ S$ [0 f+ R# O `- v6 A2 @9 {
! m' h. l& M, S# K
0 {( \# a+ q4 h5 ~- w; T 465 of 465 100% in 81s 5.72 B/s done0 ~7 n& Z V* }
4. 常用操作
' \, k$ b* ~ p+ s' ?1 `( q9 [复制
* f+ C' L, I6 r5 _, }7 G2 e% b# 1. 列出所有 bucket 4 Y# h" U$ E$ h
[root@compute01 ~]# s3cmd ls - G7 J# t5 c& d" M |+ e
2022-02-09 06:35 s3://ceph-s3-bucket9 b' _: b0 }' V+ `, V
2022-02-09 07:45 s3://s3cmd-demo
0 h+ V/ I9 n* h4 E' p
D; Z+ D4 R; T4 o: {# 2. 创建 bucket
D1 ~. h" G/ I( C/ Q[root@compute01 ~]# s3cmd mb s3://gwj-demo/1 @: K2 ^! F9 F# T& f' @
Bucket 's3://gwj-demo/' created, m) ?3 B; H- k% y' i! p3 J3 g
# A+ r4 W$ I" E6 ^: Y
[root@compute01 ~]# s3cmd ls+ n& p U! k' [# D4 H
2022-02-09 06:35 s3://ceph-s3-bucket5 K8 O; s) @4 v3 \. `
2022-02-09 08:01 s3://gwj-demo4 E7 l: S5 J0 Y
2022-02-09 07:45 s3://s3cmd-demo3 G c7 P3 ]* R4 ?* J5 ]
# C' A& \7 \6 F
, V6 B X! }' Q s/ c; E, S3 B# 3. 删除空的bucket * L: V6 |. g9 n" [1 p8 b6 Z
[root@compute01 ~]# s3cmd rb s3://gwj-demo
" {: ?3 M( q' H, o. q! jBucket 's3://gwj-demo/' removed
]5 \! _7 |- r- z( Y, C) J9 f7 d: _" a0 Z4 w
[root@compute01 ~]# s3cmd ls
$ Z! p( }* T8 ?5 X8 w% L4 h2022-02-09 06:35 s3://ceph-s3-bucket
) N( z8 f& p& P. k! [4 ^- v! Z2022-02-09 07:45 s3://s3cmd-demo3 ?, O0 Z3 R5 q8 Y
1 ^: j- }4 b7 @2 Q# 4. 上传文件到bucket
1 s* D0 n+ C# ?, y, B[root@compute01 ~]# s3cmd put ip s3://s3cmd-demo& b$ u7 v7 i% F* r3 B
upload: 'ip' -> 's3://s3cmd-demo/ip' [1 of 1]
. q; x& ~5 b. R7 ]2 { 5 of 5 100% in 0s 296.75 B/s done7 l& l3 P, P5 m3 f' ?0 g+ Q
2 [ `4 Q! T7 c, J) f
# 5. 上传目录到 bucket
5 v' b" |* K2 O% L& d[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/* O; m$ j7 p( e i4 R8 C+ U
ERROR: Parameter problem: Use --recursive to upload a directory: ./: c4 U0 V! F% N5 W$ L
( |0 H8 a( I0 J, [
[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/ --recursive( p4 `7 q/ u5 O& p3 l# d8 [( L
upload: './.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' [1 of 177]
- N6 e, [7 S2 x; B( g0 W# Q 106754 of 106754 100% in 0s 2.29 MB/s done
( j# r W. t* A2 gupload: './.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' [2 of 177]
# x; m4 K' ^3 ^1 W p* ]$ H* o1 `6 C( Q 111789 of 111789 100% in 0s 2.13 MB/s done# H: O9 s% }3 H0 R3 j B. ]
upload: './.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' [3 of 177]3 i6 H# }% y% ^1 d o7 N, f
111789 of 111789 100% in 0s 1485.51 KB/s done
3 U, R5 ~1 s3 Z: h b! M. ^! d5 E8 u
4 q# c& }, c) ^+ e( }1 i; }! `9 Q) Y& }7 A" m1 e% }7 L' j
又卡住了：
# P( d: J1 y$ w$ n# x1 y& X' R. O检查：
# y' p+ P/ ~$ _3 x3 |/ F! _9 ~[root@compute01 ~]# ceph -s
3 F( q/ p+ w3 H/ k' A cluster:
# p6 `! P# i& K9 d id: a6d0359e-8295-4e62-8620-54273462e79f0 x$ R( \; C9 S3 I) S
health: HEALTH_WARN
8 n: p" c9 s( l0 u$ r Reduced data availability: 32 pgs inactive( e& Y) H0 y& T: _
$ i3 _1 {* h) |6 x0 w- D1 ~ services:4 h8 z3 Y2 S, T6 [
mon: 3 daemons, quorum compute01,compute02,compute03 (age 15m)- N+ p5 d( k: F+ J* ]2 Z8 p3 M
mgr: compute03(active, since 4w), standbys: compute01, compute02' C" a0 `" e R0 j3 X
osd: 6 osds: 6 up (since 12d), 6 in (since 3w)
2 v) |( s; G, H& _0 E- D7 Q rgw: 3 daemons active (compute01, compute02, compute03)( u: D0 i8 s W, Z I% e( z
: z' k" L; W! N& H# D9 e8 k
task status:
# G: U5 A: K% t9 z
5 H, C h* h! O data:
7 b4 e( r2 F0 ? pools: 12 pools, 1632 pgs
, E( W5 D& o G8 t7 P$ A; j3 M- M- y% u objects: 65.40k objects, 258 GiB. B. ?2 H$ R. o4 i. B1 z
usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail
8 d& ~$ H% @/ ^/ c0 w; ^ pgs: 1.961% pgs unknown5 ]* e% b& t/ V9 i' n9 Y# E4 J
1600 active+clean( q" o0 w, v! F! r
32 unknown
/ R- p9 k9 v3 _3 Y* e1 l
* i3 f/ c0 t1 I" s io:
/ W' v& ^5 z! W9 Q client: 117 KiB/s rd, 11 MiB/s wr, 145 op/s rd, 50 op/s wr0 x$ A9 ?% L8 q' s4 L
( D, W' q8 n, S5 O
[root@compute01 ~]# ceph osd pool ls/ [) h% j" t" F5 D
images
: x% W$ q. p. O2 s7 n% V0 N; Svolumes
* l9 `% ?! e' q2 o& i( |. Gbackups
0 r" ?3 ?" h# S3 X6 O: `9 W' zvms
/ \$ _$ T: L. R4 `9 y, O+ Wceph-test2 T) D ^$ s: T- }- A: G" x( x
.rgw.root* c/ _" K# z7 B9 g: v2 }+ |: ~
default.rgw.control
* ^# m8 ?7 y% I" z4 A; cdefault.rgw.meta- z) H. m+ V7 m7 F! D/ e- X
default.rgw.log0 n% W" E+ |+ b) k
default.rgw.buckets.index% ?" O' T2 J$ Q+ _& ~0 \7 t
default.rgw.buckets.data; d+ f+ T" d$ {/ F) M
default.rgw.buckets.non-ec
) z" ?6 k6 f1 Q$ N[root@compute01 ~]# ceph osd pool set default.rgw.buckets.non-ec crush_rule ssd_rule
- t$ D* {6 w( N& G; {1 `set pool 16 crush_rule to ssd_rule$ O+ L1 \( i- K- G; W1 g. U/ O
+ I3 x- d0 T& ^ M# D' g' s' s1 Z% ~
6 k' y: n5 h: Q& Y设置完成，上传完成：
' B8 ~( ^ L; }# d7 K7 ^5 xupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 1 of 6, 15MB] [174 of 177]
4 w% D4 Y+ i0 y& I8 Z! M+ s* k5 T 15728640 of 15728640 100% in 2s 5.38 MB/s done! \+ p7 S' c- w; `! F- t: j
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 2 of 6, 15MB] [174 of 177] R* P2 h2 L: A+ f4 y; R
15728640 of 15728640 100% in 0s 38.25 MB/s done
* M0 f( i, V" T% K7 Z9 U$ g& Hupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 3 of 6, 15MB] [174 of 177]9 G# K1 R. K+ I; ^
15728640 of 15728640 100% in 0s 22.08 MB/s done
* r4 U' x$ {- jupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 4 of 6, 15MB] [174 of 177]
}8 B9 A7 D# y5 x, \5 q 15728640 of 15728640 100% in 0s 15.15 MB/s done
; C& @: t+ u, ]7 zupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 5 of 6, 15MB] [174 of 177]7 Y0 h. U6 c: X+ ^, j
15728640 of 15728640 100% in 1s 11.45 MB/s done
b/ A+ r/ [' Fupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 6 of 6, 11MB] [174 of 177]
8 j! G2 _2 D! h! {; L/ {! r \ 11599872 of 11599872 100% in 0s 21.31 MB/s done
9 g$ S8 h4 E0 N" A9 v: hupload: './monitor/telegraf_system.conf' -> 's3://s3cmd-demo/monitor/telegraf_system.conf' [175 of 177]
/ V+ ^1 N1 Q/ k9 E+ k5 P 4322 of 4322 100% in 0s 87.82 KB/s done' Y! x3 R: |9 S! ?; E2 G
upload: './osds.map' -> 's3://s3cmd-demo/osds.map' [176 of 177]' N6 F/ l7 v) V! u3 W A. y# e% F
5895 of 5895 100% in 0s 120.84 KB/s done8 S# E3 t& K; Q: T8 g1 I
upload: './s3client.py' -> 's3://s3cmd-demo/s3client.py' [177 of 177]
2 t" H; Q. x# S, X# S# Z9 j 659 of 659 100% in 0s 13.74 KB/s done
{5 @$ {* b7 q3 m. u$ L: ^, B' o4 c! ]4 X) k3 V
8 f" c4 @' D( l$ E2 A1 ? Y
3 z! M/ v& e- V4 m' j- w- `# 6. 列举 bucket 中的内容 + I' q, \6 Y6 @4 D1 ^6 ^
[root@compute01 ~]# s3cmd ls s3://s3cmd-demo/
3 P: b0 n( n5 O' L DIR s3://s3cmd-demo/.ansible/
. ~; l6 a P/ V0 _; K1 k N8 ? DIR s3://s3cmd-demo/.cache/
- ]: \$ @' Y' x DIR s3://s3cmd-demo/.gnupg/
$ K1 p1 j: {' O1 f DIR s3://s3cmd-demo/.ssh/
/ a/ g$ D6 X z9 ]) C! R DIR s3://s3cmd-demo/monitor/
9 `& e8 B2 m3 T/ G2022-02-09 08:08 25196 s3://s3cmd-demo/.bash_history
; @! X5 v. F" B5 f1 ]1 _; m$ B. G2022-02-09 08:08 18 s3://s3cmd-demo/.bash_logout, @: e5 W1 ^- H. R# U/ \
2022-02-09 08:08 176 s3://s3cmd-demo/.bash_profile
# v3 V6 F; J* `" X, Z' T2 w+ x4 K2022-02-09 08:08 176 s3://s3cmd-demo/.bashrc3 Q6 T4 x* |& f7 G6 O% |
2022-02-09 08:08 1077 s3://s3cmd-demo/.cephdeploy.conf
1 ~. C; U& X: V4 X) }. w5 N2022-02-09 08:08 100 s3://s3cmd-demo/.cshrc* q; n5 h' }/ ] L* G
2022-02-09 08:08 42 s3://s3cmd-demo/.lesshst
5 s) v& ^. y' _0 C, W) m2022-02-09 08:08 2248 s3://s3cmd-demo/.s3cfg! `( ~8 q8 p: `+ X0 I
2022-02-09 08:08 129 s3://s3cmd-demo/.tcshrc& T& T. N4 C7 E1 d) ]
2022-02-09 08:08 6490 s3://s3cmd-demo/.viminfo$ `0 M* B7 G8 R" M3 [+ {; D7 P
2022-02-09 08:08 34064 s3://s3cmd-demo/11117 X; P" `, ^: c# X( {
2022-02-09 08:08 1797 s3://s3cmd-demo/anaconda-ks.cfg# W: D3 y# k* ?5 Q8 q* B
2022-02-09 08:08 86604 s3://s3cmd-demo/ceph-grafana-dashboards-14.2.8-111.el7cp.noarch.rpm
! H) q7 a; \' I$ D8 z: ~, T2022-02-09 08:08 9625568 s3://s3cmd-demo/ceph-mgr-dashboard-14.2.8-111.el7cp.noarch.rpm" |$ {' g3 `5 f! U1 I) M( y+ T
2022-02-09 07:52 465 s3://s3cmd-demo/fatab-demo3 P: ^! f) U& L% e( E% W& ]8 n
2022-02-09 07:58 465 s3://s3cmd-demo/fstab' m0 V8 t; V/ I8 P- b' X
2022-02-09 08:08 5 s3://s3cmd-demo/ip
, K1 Z, o6 r2 Q7 M2022-02-09 08:11 5895 s3://s3cmd-demo/osds.map/ K7 A' y5 s: B8 R4 [. t
2022-02-09 08:11 659 s3://s3cmd-demo/s3client.py& L* f1 P9 [% V; M% t4 R. r
$ P1 A2 i: J) A9 u
# 7. 下载单个文件
- G% b2 r2 ^2 c0 c* S( ?( O' F[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip/ S/ k) e, |3 m
download: 's3://s3cmd-demo/ip' -> './ip' [1 of 1]" k. o7 k- p, F4 ~7 M
5 of 5 100% in 0s 396.17 B/s done. _ q, x R. q3 _1 F2 I A& I
! A6 r; A2 q. c) \5 J0 j3 |
[root@compute01 tmp]# ls
8 P+ i# V& E! y4 V3 M' e! Vip0 w0 ]9 M, f0 }+ u
2 M+ z& M! Y% b# 8. 删除 bucket 中的内容
8 L6 \; [$ H; U: J! [0 G3 M( \[root@compute01 tmp]# s3cmd del s3://s3cmd-demo/ip& F$ U5 c# `3 S2 [6 }7 [; ]
delete: 's3://s3cmd-demo/ip'
) T S8 e# I" A
8 S7 u( r0 o+ j7 H2 x5 m$ L# L[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip
; j8 P" T3 Y2 [ FERROR: Parameter problem: File ./ip already exists. Use either of --force / --continue / --skip-existing or give it a new name.9 v N' b& X' y" V
[root@compute01 tmp]# 5 ^5 @. {; ]1 [* t" ^3 w
3 r- B( Q0 T% k; v6 L# 9. 获取对应的bucket所占用的空间大小 5 }! K& S3 W X8 ]/ j! G# t
[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/
8 ?! ~' y9 |1 G" K( S/ ?5 B 104M 178 objects s3://s3cmd-demo/% r/ o9 a; b1 |3 K
[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/.ssh) Q: \7 U1 V' e2 u
4K 4 objects s3://s3cmd-demo/.ssh/ r+ l! s7 G" T: G, b) D7 z3 s
6 e" t9 V, ^- `
# 10. 查看bucket文件信息
1 J$ w7 A2 F$ ]' v8 S. \6 |[root@compute01 tmp]# s3cmd info s3://s3cmd-demo/fstab& p# o4 b/ ]5 F! o
s3://s3cmd-demo/fstab (object):6 c# H- Q* y% D/ |3 U% f
File size: 4651 }/ z; L X8 T3 b$ ?
Last mod: Wed, 09 Feb 2022 07:58:45 GMT
4 S! A: N0 O. \2 s MIME type: text/plain
/ d/ @2 C* M" w7 e/ ^" U% z! o Storage: STANDARD
Y/ b. M9 X c( I3 w+ o MD5 sum: b3b65116db83a743d6e80ae6c31896ef
Z2 t J B0 Y) g6 U K I SSE: none
+ _8 _6 w+ X+ A1 m: @! D/ ]/ H Policy: none
4 ]0 S6 ~% |# g( _; c; c CORS: none
# F N7 Y L( F( X+ x9 c' }0 t+ B ACL: Ceph S3 User Demo: FULL_CONTROL
$ R$ m: c; @6 G4 E$ y2 j x-amz-meta-s3cmd-attrs: atime:1644387423/ctime:1638501772/gid:0/gname:root/md5:b3b65116db83a743d6e80ae6c31896ef/mode:33188/mtime:1638501362/uid:0/uname:root, Q' M, \; |# ?+ T) Q2 t
( P9 E0 {1 ~5 G; U
# 11. 两个bucket之间相互cp
* u- m8 e" W0 r3 m, s[root@compute01 gwj]# s3cmd cp s3://s3cmd-demo/ip s3://test-demo/ 7 a- J K. \2 N; U# L2 ~
remote copy: 's3://s3cmd-demo/ip' -> 's3://test-demo/ip' 9 ?# g# P6 M9 P' ~0 i
[root@compute01 gwj]# s3cmd cp --recursive s3://s3cmd-demo/.ssh s3://test-demo/ 2 ^: A5 }7 Z* o1 D# x' @
remote copy: 's3://s3cmd-demo/.ssh/authorized_keys' -> 's3://test-demo/.ssh/authorized_keys'
/ b, }/ \$ `6 W7 J5 L) Rremote copy: 's3://s3cmd-demo/.ssh/id_rsa' -> 's3://test-demo/.ssh/id_rsa' / I7 t: z1 A. B
remote copy: 's3://s3cmd-demo/.ssh/id_rsa.pub' -> 's3://test-demo/.ssh/id_rsa.pub'
' K! ~7 `- ?2 y- jremote copy: 's3://s3cmd-demo/.ssh/known_hosts' -> 's3://test-demo/.ssh/known_hosts'
9 j& d5 J9 \6 B# y[root@compute01 gwj]# 4 c$ ~* ]6 L: T4 s! U' b
! Y2 k( l9 e5 Q$ B$ S. Y# 12. 两个bucket之间进行mv操作 : r1 `5 W O! h* ^: I
[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp + V) C1 E$ d- Z# g
2020-10-06 10:24 12288 s3://s3cmd-demo/.swp ! y3 U/ b, P( {1 N
[root@compute01 gwj]# s3cmd mv s3://s3cmd-demo/.swp s3://test-demo/
1 y8 a5 J. T0 H1 d0 u0 Ymove: 's3://s3cmd-demo/.swp' -> 's3://test-demo/.swp'
( k# g* z8 [. W9 {[root@compute01 gwj]# s3cmd ls s3://test-demo/.swp
0 M" I: s! ^6 A9 h& D2020-10-06 10:36 12288 s3://test-demo/.swp + {+ H; }, h! w: {. W. a1 r( i7 Z
[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp 8 [- X$ y$ L- D! s
[root@compute01 gwj]# ' b# o' g, s/ n
7 c- g' A" G9 ~7 h# n- S. @
# 13. 列出需要同步的文件和目录，但不进行同步 8 z3 {0 M2 q' q9 ~6 X |
[root@compute01 ~]# s3cmd sync --dry-run ./ s3://s3cmd-demo 9 B i* p" p A# @% [
upload: './.swp' -> 's3://s3cmd-demo/.swp' ) g9 e5 n* l+ D; u, K# h% I
upload: './ip' -> 's3://s3cmd-demo/ip' $ n4 P7 _0 v f7 w" i+ N; e' j
upload: './.cache/abrt/lastnotification' -> 's3://s3cmd-demo/.cache/abrt/lastnotification'
. t) `% `8 F+ gremote copy: 'size.log' -> 'gwj/size.log' 7 J: k# h {, T) w- @
WARNING: Exiting now because of --dry-run " s! i" R+ ?0 K& T9 M
[root@compute01 ~]#
1 i, g# L4 i/ c, {- I4 t& S
) d" I+ K( B& Z! L# 14. 在bucket中删除本地不存在的文件 8 B( [; F; ~2 g4 [8 q, f( I
[root@compute01 a]# ls
6 U: h: @3 M& m: @' b10.txt 1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt 9.txt
! G# A9 K, x4 m; O[root@ceph-node01 a]# 1 T/ w( k3 T4 z9 X+ ] U& g
[root@ceph-node01 a]# s3cmd ls s3://test2-demo/
* n& Y# J/ L7 O+ P8 ~' [: \! s; w- n* {' d
[root@compute01 a]# rm -rf 10.txt
2 K B5 |) f4 x3 @& S. M9 c[root@compute01 a]# s3cmd sync --delete-removed ./ s3://test2-demo/ $ N1 v: V! B9 B
delete: 's3://test2-demo/10.txt'
( P' d4 |2 [( d7 ?[root@ceph-node01 a]# s3cmd ls s3://test2-demo/
2 Q: n1 |( E$ e: H4 W! g, j C9 C) W2 e% n6 W: x5 w" k. V
[root@compute01 a]# , V9 [- a p4 H4 k7 L: |6 m
. |5 ?! V+ o; Y1 ~; J9 ~+ t
' D9 I! @# F& O
使用 swift 访问 RGW* x5 ?# p3 c' G
1. 创建 swift 用户 r' k8 `) u0 {/ c9 B* b4 C
复制
0 f% o* w; b0 r/ m7 G[root@compute01 ceph-deploy]# radosgw-admin subuser create --uid ceph-s3-user --subuser=ceph-s3-user:swift --access=full
; r) `; x% e. g' Z9 ^9 d' a{
, x p9 h8 m( _2 K; I r "user_id": "ceph-s3-user", . v1 X& r# s6 I* i6 f
"display_name": "Ceph S3 User Demo",
, s, _! V, i5 f" O& \ "email": "",
@0 x& @. T8 k$ x "suspended": 0, / k" x5 S9 j2 b6 W( h- F
"max_buckets": 1000, : D8 B$ J: A$ h7 H
"subusers": [ # P4 f' f- p4 x" P! ?
{ 7 j7 s; q. Y% K; x( T
"id": "ceph-s3-user:swift",
# d1 M: H" C' X6 n "permissions": "full-control" ; D3 d c G# x
} ( t/ M) t7 Y- Z, t/ @: N4 U! T
], & e: S1 h7 u2 o( ?
"keys": [ 5 }. x h/ u$ |4 k3 |
{ 2 |$ J+ N# M3 d' `
"user": "ceph-s3-user", 8 s6 ?8 _. u0 Z6 V
"access_key": "V3J9L4M1WKV5O5ECAKPU", 8 ~ [2 m; E0 ]3 Y J4 N1 L
"secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw" : d$ Z& V2 k! U" z; P1 Q. M
}
* R' ]% }9 [9 i, E( @1 x1 Q& ] ],
% v' f% h+ B, r3 b "swift_keys": [ * A; g4 f4 C( {8 Q- `3 Z" m" p
{ : m* @2 B' ~ Y, [) K& n
"user": "ceph-s3-user:swift",
5 J( ~2 Z" ~+ D* x' `( T7 Q( @ "secret_key": "ZIOOU8Xcfe3m6ZZapK5P2rU0GGPaiS31chy9yvMW" , a2 a- @4 ^7 j7 _7 m
}
( @9 M4 L+ P0 H. z: R ],
0 [7 r- I% H0 }) M "caps": [],
; X+ e' h8 M. U8 u: s "op_mask": "read, write, delete",
% _# u W: {: @2 K) h2 {; d "default_placement": "",
2 ]9 }) j! E5 c$ n+ ^ "default_storage_class": "", ( M: F* y+ m; a C
"placement_tags": [], & P) C0 |9 K0 W7 [! y1 n L M
"bucket_quota": { ; B$ u: H+ J! G+ o9 m. y" P5 ^
"enabled": false, : x8 W3 F3 ]4 F
"check_on_raw": false, ) L q8 v6 X9 h* o
"max_size": -1,
/ H( [' J' D6 |& O1 E7 r "max_size_kb": 0, ' O6 ^8 f" c% V( y' ]! P$ d
"max_objects": -1
5 o! |5 G _& @+ X9 p },
% a/ u3 u# e6 G7 ] w& _+ \. u "user_quota": {
+ G5 |0 k7 ?0 @" D6 } "enabled": false,
& L- N0 o" Z- A# K "check_on_raw": false,
@$ w+ S4 P2 E2 B6 \! @8 V "max_size": -1, $ i+ |. w; n z
"max_size_kb": 0, 2 ]6 w8 J7 s: ]' i* N2 d* P5 E) ~
"max_objects": -1 : K6 p: U9 L4 b
},
% }4 G6 \8 U- g1 v "temp_url_keys": [], 1 t/ @+ S* f' n- _- ~: O
"type": "rgw",
0 M2 _" H3 j& M" A6 ~( i "mfa_ids": [] 1 p6 C5 n/ n; x4 U7 J0 |
}
L# e; `0 ]& I
& Z3 B+ X+ C4 Q+ i& B[root@compute01 ceph-deploy]#
- M$ y/ b3 ]$ U% E; Q+ F7 x
# e+ M" t0 S; K* _3 C: V
- i l$ f; ?9 ]4 h2. 创建 swift 用户的secret" n. }" b1 \# O3 A
复制" J) Y6 M0 ~5 q8 i/ A+ L
[root@compute01 ceph-deploy]# radosgw-admin key create --subuser=ceph-s3-user:swift --key-type=swift --gen-secret ; w/ ~5 p- o* U9 W) M
{ ) ]* ?+ Z: t0 k
"user_id": "ceph-s3-user", % t; l6 j m6 L& L+ O, k
"display_name": "Ceph S3 User Demo",
7 X2 C- A1 m0 @ b "email": "", ) {8 `7 ^0 L @
"suspended": 0,
* B4 Q7 S8 K3 V# e# k" h) u2 \ ~ "max_buckets": 1000, , R, M2 X$ p9 {6 N9 W9 w) V
"subusers": [
+ O) C" [# A9 m) H0 ? { ) D8 Z6 _- b6 F4 H, K+ ~: F) I4 Y
"id": "ceph-s3-user:swift", & Z' p4 ^. k# q- H
"permissions": "full-control"
) o$ ~# B. T1 `! s: _* Q }
& }6 z" F& y- Z. q8 Z* u ], 2 v# j- ~# L2 i/ j1 t j' d
"keys": [
! }1 s' a; F6 o { 3 i. [$ b+ `6 U/ ~% X" c
"user": "ceph-s3-user",
2 h+ L& u j9 B- A8 | "access_key": "V3J9L4M1WKV5O5ECAKPU", 2 q1 e6 `1 R* [* c' K, @
"secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw"
8 \* J* G/ X, `% r7 A4 x }
9 \' Y L: r8 y$ ?/ p l' j$ K ],
( ]2 w1 f v9 b0 u( o0 d o& j "swift_keys": [ * I6 U) B" C) I
{
* R; H% i9 j- d. A, S' M4 o! x/ K2 C "user": "ceph-s3-user:swift",
- @1 U5 S) j; p$ D "secret_key": "0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE"
' B1 O2 C5 ?3 q0 m2 t4 i }
# E; \" l% f2 B8 S ],
0 E, s6 l$ G8 u, F/ w! Z "caps": [],
& `# z# T3 n/ J& {" e1 Y* S! o "op_mask": "read, write, delete",
, Q, _, h; N0 M1 s/ ]; d! r "default_placement": "", 3 D' d" @, A S( L8 M
"default_storage_class": "",
' a p) P, J8 J/ g! @/ y% U "placement_tags": [],
% Z0 d- X* `/ x$ i. B+ u( Q) I "bucket_quota": {
5 B& o- \! k) r8 h3 v" } "enabled": false,
2 q* ~2 D" A- K8 L3 T. W1 U1 g) s "check_on_raw": false, ( e% y0 g# r( T' A
"max_size": -1, : a+ I# Z) u9 H" v
"max_size_kb": 0,
) h4 A/ p" A2 S1 H "max_objects": -1
7 _2 k! y8 c+ H( t# `6 m }, G5 \' Q. n) n, B' J5 [
"user_quota": {
# j* ]: o8 Y# Z4 O9 U& p "enabled": false,
' A: E' `; y% Q6 N3 e$ N "check_on_raw": false,
7 W% B% A8 O/ [. u "max_size": -1, : K( C d1 G# D
"max_size_kb": 0, Y- A" N* q8 V u1 R
"max_objects": -1
% V, S/ y, L* `+ Q& ^ T }, 0 b/ E/ X( x- z! Q0 \9 X& f
"temp_url_keys": [],
5 o0 x! }" V9 v: t$ } "type": "rgw",
1 [- [0 _! X# { "mfa_ids": [] # b7 l X Z; g- E; T: l* j
} ! V( B8 Y: t# Z& a) _: [. x
) B- G/ R8 Q% ]% f" J[root@ceph-node01 ceph-deploy]# 3 P$ E% y$ e) ?' i
* R$ c3 ]7 T- D# D! }. K4 J, R8 Z; h& i5 f2 t
3. 需要使用 pip 安装 swift 客户端工具- c; R% ^9 |2 f/ q) u
复制! r5 R! j4 E' H5 y1 A4 u
# 注意如果有pip的话，就不需要再安装了 ! e* U' D; m4 H& `& X
[root@compute01 ceph-deploy]# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
) R E) [) A* h/ \7 [& G1 c/ |。。。
5 W! t3 m2 W3 c% S[root@compute01 ceph-deploy]# python get-pip.py
. _9 l' G& b2 y$ f。。。
* z9 v% H. ~& J& C% h[root@compute01 ceph-deploy]# pip install python-swiftclient , t# W/ x8 ?4 k% ^
。。。 # _7 @# Y* v2 ~( l5 T
; ~* q$ [" D, K; Y9 @0 S: S
e, N5 c& b# ~# s
4. 使用 swift 命令行工具
( J1 P) m. x# h; B复制
6 O$ D4 g% i: Q$ O5 @[root@compute01 ceph-deploy]# swift -A http://100.73.18.152/auth -U ceph-s3-user:swift -K 0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE list 8 h7 r& y7 q- |& V) ~
ceph-s3-bucket
2 y0 V& @& H0 m* q) e9 es3cmd-demo ! l0 [3 f5 C3 U6 E y
[root@compute01 ceph-deploy]#
6 m& r+ t9 c; p+ D) b5 t5 T7 A7 R) _( I! D
8 C! D/ l: f3 q/ L T5. 配置成环境变量的形式
3 v, s/ x6 M; n7 M a7 S4 `复制
: a0 Y1 `$ z$ }. l[root@compute01 ceph-deploy]# cat /etc/profile % t- j4 L% h% L) V3 U* ]# K
。。。
7 X* Q, T) K% U4 I; s( F6 B' Oexport ST_AUTH=http://100.73.18.152/auth ; o4 a4 L ]. Z
export ST_USER=ceph-s3-user:swift , v% e9 S. Y) b3 y& h. W
export ST_KEY=0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE
, Y5 i) i4 M! Q[root@ceph-node01 ceph-deploy]# source /etc/profile . V9 }+ i4 l `$ m7 b
[root@ceph-node01 ceph-deploy]# swift list
7 N' G9 U8 K+ j) n# o8 Sceph-s3-bucket
8 p y% f2 D( \* X0 }s3cmd-demo
' L6 j! V" Y: w! P" S3 G[root@compute01 ceph-deploy]#
- T' p& Y5 u" L0 B/ z
0 r# A& T6 Y* T# F# C' S: U: k$ T$ f+ q4 |9 c) s2 @7 d* Z
6. 创建bucket" Y5 `. h" r& R( x; F, }" V
复制- R3 q0 e! `8 F% v3 S' |- F! _! t$ k
[root@compute01 ceph-deploy]# swift post swift-demo # Z; w) |) D. q" [& r1 g
[root@compute01 ceph-deploy]# swift list
`# p' q( [5 |ceph-s3-bucket
^, Q# T# h& vs3cmd-demo ; z- `1 `4 m) x4 L; C, F) Z7 ^/ s
swift-demo ( K- i. L/ j. A7 E7 e0 A# ?. s# H
[root@compute01 ceph-deploy]# / E {$ S1 m, H0 w& R" X
) x0 o7 ~$ @( p$ b7 E, r- S0 o
9 t0 D6 r4 u& s- T* k7. 上传单个文件测试
% g2 _2 z b) O$ c复制$ J# }7 ?+ X; B
[root@compute01 ceph-deploy]# swift upload swift-demo /etc/fstab : Y0 ?6 ^! u: ~- I- T
Object HEAD failed: http://100.73.18.152/swift/v1/swift-demo/etc/fstab 416 Requested Range Not Satisfiable
% m) Q4 ]. I8 e* Q1 f. m) }[root@compute01 ceph-deploy]# ; q) X Z5 |' \7 [. p8 n* i4 C5 A
: c% F8 F& e g
8. 上传目录测试
9 {- @3 y, X% U8 B复制% A, i, c! S! b3 h3 R; Q- [
[root@compute01 a]# swift upload swift-demo /etc/fstab : K0 [# w6 r9 v* \
etc/fstab
' \4 K6 L8 m. Z1 `1 k[root@compute01 a]# / b5 K4 g" H# d/ P9 A8 w
~& {2 o: W' r; v( A3 T. c8 M( U% Q4 B9. 常用操作) i% [# T) p' C6 j
复制8 _, T8 n/ b) S# W+ [1 I
# 1. 列举所有 bucket * R* }( K8 c# J) T- t, ` e
[root@compute01 a]# swift list 1 s; ?! j d# C' ^" T
ceph-s3-bucket 7 r7 D. X& I$ u/ j" X
s3cmd-demo # n. i& }) q( v: y% ?0 X, Z
swift-demo ! x1 ?& A& Z7 }( a) F0 N9 G& e8 t$ a
test-demo % Y+ M4 u8 g1 p+ X4 C. g
test2-demo
0 x f8 `$ a3 i[root@compute01 a]# : S: _5 c. e c% i* ]+ ^8 Q
/ j. z# ~8 N; d! h( L5 a! p# 2. 列举所有 bucket . ]' P0 a0 d) c9 y7 b A* n& d% x
[root@compute01 a]# swift list --lh 9 z) E, v6 H+ S# W5 W5 u2 b
0 0 2020-10-06 04:13:10 ceph-s3-bucket ; S, \! B: F6 J. [. j* T$ g
37 3.6M 2020-10-06 04:34:49 s3cmd-demo 7 I& r5 ?8 x& z- h6 D. n" q
2360 33M 2020-10-06 08:07:55 swift-demo ( q! D8 @9 ~% H% @
7 16K 2020-10-06 10:32:02 test-demo
6 g6 a8 H, S0 ^, j8 Y, ~1 c: S* } 9 387 2020-10-06 11:17:00 test2-demo
# T. Z: R1 k8 {' p. p9 N 2.4K 36M 8 J I4 b" [8 ^% \3 \
[root@compute01 a]# 2 h4 Q: N2 M0 U7 u8 |# S1 p
3 [" y" ?" o4 Z E; T$ H6 ~( Q" N# 3. 列举单个 bucket
! T$ t# g# G6 ?3 z[root@compute01 a]# swift list swift-demo
& `% q; g/ t9 b$ y2 v$ t; z1 C! O# J+ ~# I9 U/ H7 p+ Z
# 4. 上传单个文件到bucket
, ~& r+ b3 d; m K- m# w[root@compute01 a]# swift upload swift-demo /etc/fstab
4 u! T) B9 m* ietc/fstab
3 e5 R, I" U1 w \" w# X8 P; w[root@compute01 a]#
5 T2 f. E+ G: G7 F1 N0 r& n0 i. C3 L& a; C# ]+ J; }
# 5. 上传目录到指定的bucket " a) @: L% C. C% S: F8 U2 a9 K8 g
[root@compute01 a]# swift upload swift-demo /etc/ 6 F. u7 O+ l) j1 p5 k' `, @
$ s6 N8 Q2 [- @2 {5 Q
# 6. swift 状态信息
9 A; J I* m9 G; Z7 E[root@compute01 a]# swift stat % T h3 a8 V0 E4 E
Account: v1 ' o" L8 {" u6 Z/ p) h+ X
Containers: 5
8 O9 t! `. j y6 N Objects: 2413 # T( D3 ]2 w9 q' E7 L
Bytes: 38701415 1 a+ f. z4 p U( i
Objects in policy "default-placement-bytes": 0 4 V N m @9 H: S; c9 W
Bytes in policy "default-placement-bytes": 0
0 g! r [( V9 M1 j+ x Containers in policy "default-placement": 5 7 X+ v! H) |( K) m, F; Q- f# u
Objects in policy "default-placement": 2413
: z5 t# @$ u5 F' n2 b. k Bytes in policy "default-placement": 38701415
/ N4 M9 T/ B; a% x4 I X-Openstack-Request-Id: tx000000000000000001302-005f7c5afd-a638-default
: R" S6 F9 A5 h/ I! g X-Account-Bytes-Used-Actual: 45948928
! [! B% `" B4 u/ a5 s) U1 l/ J X-Trans-Id: tx000000000000000001302-005f7c5afd-a638-default
) e5 ^5 \) C( K4 _" j( d( `7 Q X-Timestamp: 1601985277.38095 4 F: u: Q7 K3 ?9 D/ n) i
Content-Type: text/plain; charset=utf-8
5 W* P8 ? { d% [- U& |/ \) ` Accept-Ranges: bytes 4 o7 k5 l# z5 ~1 W: s) h6 V6 x$ \
[root@compute01 a]# # G& a1 @2 ]2 @( o# b
' U6 @3 [/ `& T0 m# 7. 创建 bucket
+ w, u8 A# e, B$ _( L[root@compute01 a]# swift post swift-test
) ^1 A9 A, x3 C. ~[root@compute01 a]# swift list + u3 J( U; P: E) N$ ]) ]
ceph-s3-bucket 9 _" |: v3 Z5 }( w2 v* S+ H
s3cmd-demo * `: |& \. \3 j. Z
swift-demo
8 Z T' @3 i9 `! ?2 M% D: s8 Bswift-test
2 f: l9 @0 J0 C1 Ltest-demo ( V( X) v& M9 j4 c
test2-demo
4 }! o& K3 | ~- L) Z8 e$ Q9 o[root@compute01 a]# : T6 u3 I# h4 K U" I" H6 Y! e
; u( |0 p9 Z3 q: Y# W
# 8. 删除 bucket 4 r2 @! o% Q" V( y" a9 @
[root@compute01 a]# swift delete swift-demo
# R" P# Q0 r/ r" V$ O9 Q- v. f1 h5 D: n$ M5 E( g7 H
# 9. 删除指定 object - p% ]8 i( X, q6 t( {1 W( n
[root@compute01 a]# swift delete swift-test root/a/1.txt $ f( I1 H; k* h: c' T
root/a/1.txt 9 q7 v! S! a4 Y, H* P1 L% T
[root@compute01 a]# ( H/ V5 H4 _# a! v: W
2 q, C1 C m( z/ Y( F' z# 10. 上传大文件时可以使用-S指定分片大小
! H/ r9 Z0 _6 U9 ^[root@compute01 a]# swift upload swift-test /home/log.txt
7 F8 C6 M$ x/ }( q+ n1 Ohome/log.txt 5 [' a& ?, h6 g
[root@compute01 a]# swift upload swift-test -S 102400000 /home/log2.txt , a0 g1 p' f+ p. @/ p
home/log2.txt segment 5
/ l, k. S+ @0 H: M$ Uhome/log2.txt segment 3 & o; `0 ~' @) k# F1 r2 k! f" y
home/log2.txt segment 1
! A3 i$ B/ X/ R- O5 V* k" m4 Zhome/log2.txt segment 0
2 x( |8 j4 N' i2 e& E4 v, b, hhome/log2.txt segment 2 0 U, o% U0 ?# g3 m0 M; J
home/log2.txt segment 4 1 U! R/ L# W6 s8 A* H6 h3 z
home/log2.txt * H/ n7 q/ K# `7 l9 |* Y. b
[root@ceph-node01 a]#
\2 }' D9 |1 C$ G0 |0 z( k. ~; N; U
总结
+ E/ t7 ~8 M5 _2 Y从概念、安装、到实践使用，简单介绍了对象存储命令行工具。: o, E* c0 j$ x, b D9 i
! L) X. j" n. `2 W
( y2 O1 \( c {9 }7 I! h, m
+ p! B6 c3 E2 N4 R
|
|
发表于 2022-2-9 15:15:20