|
|
Ceph 对象网关主要是由 radosgw 接受用户请求，然后与后端librados交互;它主要起到承上启下的功能，对外它能兼容两种接口，一种是S3，另一种是Openstack Swift 接口，这两种接口，都有各自的用户认证机制，所以 Ceph 也提供了一套独立的用户管理机制，这套用户管理机制能同时兼容S3和Swift接口，最终数据会落在 OSD 上面，无论使用 S3 还是 Swift ，它们落到 OSD 上面，都是相同的名称空间;所以你使用 S3 存储的，也可以使用 Swift 进行访问。这是对象存储的一个基本架构，所以如果要使用它，我们需要部署radosgw，才能访问到集群，默认是没有安装的。4 r* d$ L5 |8 x& T, T
# \, D, a2 I7 [5 W+ y
bucket 是什么?可以理解为装载对象的容器，它的后端是无限可扩展的存储空间，并且具备安全可靠性。Ceph 对象存储它后端是借助 ceph rados 实现数据容灾的机制，那么他能提供哪些功能呢?9 e3 D: ?1 x6 ?& I& k+ w' w
- P* W2 C: z. [2 N- A* g3 P基本功能. V9 P( C8 Y2 ~3 r2 |
RESTful Interface # RESTful风格的接口，实现上传下载及管理功能;
! E) o0 p9 m# TS3- and Swift-compliant APIs # 提供两种风格的 API 接口，兼容 S3 和 Swift;( d4 J' y7 _& S' l
S3-style subdomains& @: H$ L. W$ @0 X& f1 S3 L% O
Unified S3/Swift namespace # 扁平化、统一的的S3/Swift的名称空间;
; C1 ?, s+ x/ p& GUser management # 为了安全性，也提供用户管理，可以限制对象是可以公共访问，还是授权访问;
7 ~7 ]# X5 ^3 s" S% c3 T ZUsage tracking # 追踪用户使用情况 rados df! C: h9 n; w; i/ d
Striped objects # 支持分片上传
; ~6 D4 P; ]3 m; T: e) nCloud solution integration # 支持云解决方案集成
" z0 v! W2 U* P+ T8 U, CMulti-site deployment # 支持多站点部署
! Q, T0 F; y, c* W7 G$ \Multi-site replication # 支持多站点复制- ^) W. P9 A3 k) D
安装 RGW A5 G- U1 S J6 P, ^
1. 软件安装
% m* d+ T* o% j/ Q; W- w复制
( o! F T3 _8 Y1 \[root@ceph-node01 ~]# rpm -qa |grep ceph @4 v4 Y0 x4 K6 Q- s2 p- W
ceph-base-14.2.11-0.el7.x86_64
6 T/ y* G0 a" Y" ?9 i7 ~( c' B cceph-mon-14.2.11-0.el7.x86_64 3 ]. Z' E# Z4 }. \
ceph-deploy-2.0.1-0.noarch
5 ]( E7 e6 F$ W, g( \python-ceph-argparse-14.2.11-0.el7.x86_64 9 W# A1 x& i3 Q: ~
libcephfs2-14.2.11-0.el7.x86_64
7 p' Z" n4 [$ Sceph-common-14.2.11-0.el7.x86_64 ) {3 E2 D/ @' J! y+ I5 P7 y
ceph-selinux-14.2.11-0.el7.x86_64 3 a6 \4 D8 t8 H" ^( j2 z
ceph-mds-14.2.11-0.el7.x86_64 & W, y& E9 d0 A8 V
ceph-14.2.11-0.el7.x86_64
% H( x, ?2 S6 J' [python-cephfs-14.2.11-0.el7.x86_64
6 S: b3 E$ W0 A1 V6 K1 Oceph-osd-14.2.11-0.el7.x86_64
4 {+ ?, Y$ i6 q+ Iceph-mgr-14.2.11-0.el7.x86_64
/ I7 s1 @. W$ p' ]3 A% {+ p$ Jceph-radosgw-14.2.11-0.el7.x86_64 # 直接使用 yum 安装即可
+ o8 p( B2 q8 s: `3 M[root@ceph-node01 ~]#
& P4 j3 v) M9 D( Y1 t- J9 y7 p' f X8 d$ U+ }$ Z$ m% c) E* ]
; f0 H$ E: h( H/ P+ i; H2. 启动服务，默认启动在7480端口
5 R" S5 f1 x& F% L, U$ m d复制
& ]2 g" q- N+ F) K; D" W[root@ compute01 ceph-deploy]# ceph-deploy rgw create compute01 # l7 B1 _) k7 O; Q( C; R7 ?! \
5 Y+ R+ m. ]& H
3. 服务检测
o. {' w6 d$ v) \, v. a复制6 y5 E2 c a$ L# p, `4 J* a
[root@ compute01 ceph-deploy]# systemctl status ceph-radosgw@rgw.compute01 1 d# a, H: y4 A$ f6 [; T& q/ y% R
● ceph-radosgw@rgw. compute01.service - Ceph rados gateway
, Z' e/ N0 W2 J$ g8 G$ o Loaded: loaded (/usr/lib/systemd/system/ceph-radosgw@.service; enabled; vendor preset: disabled) * w6 q6 `* A& s5 C) |, x {' i
Active: active (running) since 一 2020-10-05 20:34:36 EDT; 14s ago 1 w- _7 n) k( U* z
Main PID: 33574 (radosgw)
. [0 H. w: }: y3 V! _2 e CGroup: /system.slice/system-ceph\x2dradosgw.slice/ceph-radosgw@rgw.compute01.service
, W2 y" H, X5 n& s, X └─33574 /usr/bin/radosgw -f --cluster ceph --name client.rgw.ceph-node01 --setuser ceph --setgroup ceph ( B: |2 `- v& ]& J
& a5 a' n; t$ K) v
10月 05 20:34:36 compute01 systemd[1]: Started Ceph rados gateway. ; q3 x/ V, f5 l* y" A# e
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:13] Unknown lvalue 'LockPersonality' in section 'Service'
2 Q( C1 S4 [1 s$ i- }10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:14] Unknown lvalue 'MemoryDenyWriteExecute' in ...Service' 0 A, h5 a% h* \+ B
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:17] Unknown lvalue 'ProtectControlGroups' in se...Service' 6 _, _" V& L5 P4 e/ P' w6 y B
10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:19] Unknown lvalue 'ProtectKernelModules' in se...Service'
& ^% d) j% \ s' `- y10月 05 20:34:36 compute01 systemd[1]: [/usr/lib/systemd/system/ceph-radosgw@.service:20] Unknown lvalue 'ProtectKernelTunables' in s...Service'
. a6 ^8 p4 O$ R0 }9 PHint: Some lines were ellipsized, use -l to show in full. % S7 J& J; q* f: t7 a* w" r: z
[root@ceph-node01 ceph-deploy]# netstat -antp |grep 7480
3 w3 l: I4 d3 N% Ytcp 0 0 0.0.0.0:7480 0.0.0.0:* LISTEN 33574/radosgw ! Y! f2 c) }7 G
[root@ceph-node01 ceph-deploy]# ceph -s
' i" Q* P u0 K/ S cluster:
1 |9 g! I/ Q% u, U: p id: a6d0359e-8295-4e62-8620-54273462e79f% p/ T# p3 P: R2 y
health: HEALTH_OK
, k K7 c, [% G! F
9 p% h2 l: k+ }- k# w services:
* ]) d {% b: p9 L4 z- O4 i mon: 3 daemons, quorum compute01,compute02,compute03 (age 12d)3 S; a# t5 f) K% ?3 }5 \* h
mgr: compute03(active, since 4w), standbys: compute01, compute02
( b8 }4 o& ?( k+ v7 E. t7 K+ O% p0 E osd: 6 osds: 6 up (since 12d), 6 in (since 3w)% Y+ G" T" ]* ~5 w' Y- \ C4 c
rgw: 3 daemons active (compute01, compute02, compute03)
7 J3 N7 T0 O$ C) p" y
2 y+ `1 I: }6 N$ _+ x2 T4 d7 v task status:
( u3 \' T( ]5 p7 V
- ^# [( @( ]! Z$ f* K data:
1 ]) p& o( w9 t0 z9 X. e5 w; j* b pools: 10 pools, 1568 pgs
6 F0 u2 R" X$ }: \ objects: 65.20k objects, 258 GiB+ T$ K; C9 g4 W6 j0 H
usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail8 B9 X8 g1 p. h
pgs: 1568 active+clean% U- {7 w+ Q* [
/ d$ [3 b4 w! `
io:5 Y2 t% T0 C9 r' i
client: 0 B/s rd, 15 MiB/s wr, 10 op/s rd, 51 op/s wr
( B m, q0 A4 A( l) D: U$ k8 |5 l; O: k$ |) \* x% j
1 i5 A$ K) S# C. c
% x/ `+ A" R' l# g5 J# _/ K$ X1 `$ Y+ R
4. 首次访问服务
, m/ `, j {+ K$ U2 ^' t' J复制
9 `: b) _' J0 ]4 H" X[root@compute01 ~]# curl http://compute01:9000/
' r( `( O) |9 B, T& J) U<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>
" P% B$ L2 x8 F[root@compute01 ~]# / I, `* Y: K- H
2 Y! t) d* u# ~% f
提示错误，使用了 anonymous 匿名用户请求所有列表，到这里说明安装完毕了;
- c! R8 c& B8 ]: ` X$ q- V; S, z( E& K& e' c9 U
5. 修改 RGW 的 默认端口 7480 到 9000
9 g+ N j* ^) b* x, {- d复制
& F- I1 W7 d2 `3 s4 F$ y. O[root@ceph-node01 ceph-deploy]# vim ceph.conf
: ^4 ?9 r/ {* P, c! @- {。。。
# u+ M( m U; c6 d# I- l* Q/ u x- X' B/ Z7 n- D! b
[client.rgw.compute01]
- f; l% Q0 j. f; d4 s% @; c. O! Uhost = compute01: V5 g! h- k t: s5 |; d
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring
4 g' T( j3 |0 Rrgw frontends = "civetweb port=9000"
, G2 E* I0 ~& }) d9 W) X# P7 r/ Y) ?rgw print continue = false
- m- q$ A+ i; @* x2 V% y, A5 G& |0 Y
[client.rgw.compute02]: O" A: L! G# A" N# V
host = compute02
9 U+ I# C/ I* U- I' Krgw frontends = "civetweb port=9000": t$ N, D' n+ p& D* b
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring
2 x! C4 L) s7 @: Lrgw print continue = false/ ~! R: u n- e- R) |0 z
( R* j7 |8 v$ q, \$ @
[client.rgw.compute03]) }& o, H# X3 H0 _, W7 f* g
host = compute03
$ ?4 t% N& `5 Z' A6 s" E. nrgw frontends = "civetweb port=9000") [, {3 r& C% L" T( l @5 ?
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring3 h# j3 c, }9 `4 |) r* F
rgw print continue = false% X! v. U7 b9 N# l: F
. w: W# C: u4 \0 _0 Y$ Z# s$ F0 l" P9 V! R4 y3 W) d3 C
修改配置文件ceph.conf，为什么修改这个文件呢?因为后面添加节点时，默认是copy的这个配置文件，修改这个配置文件，可以确保集群的唯一性，下面推送配置文件到所有节点;
0 g3 w3 T$ F7 H5 z9 @ ^ k6 F v8 g
复制& p/ T3 U. P& Q8 y3 T8 \( | Q
[root@ceph-node01 ceph-deploy]# ceph-deploy --overwrite-conf config push compute01 compute02 compute03 ) [. J5 m# P* J. P8 w
. l- Z( ^; Y; N' B: g" j# w7 @' M--overwrite-conf 注意需要使用这个选项，否则提示无法覆盖;* C; L% K* o2 r2 m. c7 n
* b9 ^, D) t1 b1 c
6. 重启服务
3 l7 N1 M* w- N2 L( ~复制+ v" ?; P, Z" Y' j8 w
[root@compute01 ~]# systemctl restart ceph-radosgw@rgw.compute01.service
& A! y" m* E+ X% p1 h' P3 h7 X[root@compute01 ~]# netstat -antp |grep 9000|grep radosgw & J, {" C( g' \( m Y
tcp 0 0 0.0.0.0:9000 0.0.0.0:* LISTEN 2758818/radosgw 7 `( s8 v4 u5 l) e, y
[root@compute01 ~]# + ?6 C5 y% x5 M5 C& o
7. 验证 80 端口. B* N* u5 V& A+ a: M* C
复制
$ h9 u' e) _6 A) o5 a[root@ceph-node01 ceph-deploy]# curl http://ceph-node01/ 5 M# E% k1 i0 l* P" c9 e: e
<?xml version="1.0" encoding="UTF-8"?><ListAllMyBucketsResult xmlns="http://s3.amazonaws.com/doc/2006-03-01/"><Owner><ID>anonymous</ID><DisplayName></DisplayName></Owner><Buckets></Buckets></ListAllMyBucketsResult>[root@ceph-node01 ceph-deploy]#
1 H. u3 }6 h* Z! k! ]
& g0 A& i9 L! z6 y0 S, \6 _- Z6 p' @8 \+ @7 P7 X
直此，RGW 服务部署完成;/ j1 X0 \- l* w, `. `" t
" A% ~8 @8 a) ]9 g9 |使用 S3 访问 RGW a# Z5 @ q/ u9 T; C( H# z
1. 创建 s3 的兼容用户
" {& H+ X7 j2 g, v M/ x1 m复制7 Q) h: \: h! V8 F9 E8 A2 G6 O
[root@ceph-node01 ceph-deploy]# radosgw-admin user create --uid ceph-s3-user --display-name "Ceph S3 User Demo"
+ ^' t7 O( X0 ^) q{
4 r; p! \0 o0 e "user_id": "ceph-s3-user", 9 `6 a3 O" s6 @+ d8 A
"display_name": "Ceph S3 User Demo", , t- l1 K( L% K+ d* L6 w3 a2 v3 \
"email": "",
, ?- V7 Y* m; V: C4 m "suspended": 0,
$ Y0 h% B( e/ v' o "max_buckets": 1000,
6 b5 l2 }! b. R- P, P( c/ j i "subusers": [], 7 V+ s4 O% z7 K& Z5 P' l
"keys": [ / f8 f% P. ]: ~7 T# f
{ - H/ J- N7 d9 ?" r0 R6 \: Y
"user": "ceph-s3-user",
# e8 L: s' U* \& S f4 e- ^) h "access_key": "V3J9L4M1WKV5O5ECAKPU",
' P+ I# q8 ]. d+ S% _ "secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw" : W* m6 }: y% h7 C( a9 t& h' y
} T, X- t8 [" j& N! D, V
], 9 D6 }( ]. ], x
"swift_keys": [], . I% G% h0 i5 ~" ]9 M
"caps": [], ' G: w( n, S" g1 u1 T1 s0 K
"op_mask": "read, write, delete",
) Y4 F6 }, S4 y$ @ "default_placement": "",
, w @2 T- p% v! G8 v8 U: X' @' y \$ l" l7 P "default_storage_class": "", . U/ H* q0 {) w2 N
"placement_tags": [], 6 B: ~4 \- U# G
"bucket_quota": { 4 C9 H% ?- H7 F! W
"enabled": false,
3 s* S7 `% `: b5 W "check_on_raw": false, , X" r) V" a: q( P/ R" C ]
"max_size": -1, 0 `( ]* d& y0 z
"max_size_kb": 0, $ p8 B. w' L6 E) u
"max_objects": -1
. g s8 s# n. w6 M },
1 R$ f N+ J2 o) u [% E8 K "user_quota": { 3 f/ p5 {. z t- F' D$ \7 H
"enabled": false, 4 W& |5 r5 ^( L& U& k+ X+ ?
"check_on_raw": false,
3 _2 z) z0 B1 P6 X0 w "max_size": -1,
/ d, | t- s! @( |. j$ G7 d6 C "max_size_kb": 0,
5 }* @: C( }8 b6 { "max_objects": -1 $ _- |2 ^, p, c+ W
}, 5 V) m1 B, \+ w/ h% j4 K- U/ q
"temp_url_keys": [],
: ^( h- F3 k# j+ U. | "type": "rgw", * H9 ~( w/ W! h- \9 T
"mfa_ids": [] 2 U5 S7 y" u6 t$ {( { R) c
} 9 @2 Y3 X$ ~" q) s9 g
. S* O3 X- L2 R( E. o0 a[root@ceph-node01 ceph-deploy]#
: @$ l% |0 d0 w$ `$ ?4 l: U) b% N2 L4 t) c* J& J& f* K
9 L. f" T; q+ j
注意上面的 access_key 与 secret_key 很重要，要记下来，以备后用; 不记录也没有关系，我们可以使用以下命令查看;1 k# P( ] k1 I. h# N0 r' g, h
% z" \0 f: V, ?
复制- n5 l' a$ j7 G$ e
[root@ceph-node01 ceph-deploy]# radosgw-admin user info --uid ceph-s3-user
; j6 A$ l4 g8 Q& u1 \8 K1.
3 S9 p4 `2 w9 T: R, ^; c( T( Z' @2. 使用 Ceph SDK 访问 Ceph 集群 I6 ~# m3 `: y- l
官方SDK 使用说明：https://docs.ceph.com/en/latest/ ... g-s3-api-extensions
- [% M+ A+ @# e$ d6 C
' g0 M- q- U7 Y1 M$ f. _& n复制8 g& N, E3 O. a# S$ J6 m
[root@compute01 ~]# cat s3client.py
! X: _- J% H; H1 d4 f, Z' F2 U# mimport boto
0 @, |% S/ [2 {. z) |' A' Bimport boto.s3.connection ) U5 k8 y& R r& t' d! R% u
access_key = '8Y28X33O2YJKI3H6B1WJ'
2 i+ t, j/ ?- V. I, jsecret_key = 'ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y'
/ A6 u1 X! J: a* d$ I8 \8 s0 o. W8 E: u, z/ X9 z' Q
conn = boto.connect_s3(
( o: d Y" s( x8 X* U ?) F; P aws_access_key_id = access_key, 4 W" K3 u7 Z2 Q. _. v9 O
aws_secret_access_key = secret_key,
3 u, c; Y4 Y, U' _" M8 Y0 ^ host = 'compute01', port = 9000,
4 V7 v- N# Q N' t* ^ is_secure=False, # uncomment if you are not using ssl
2 j/ ~) H" r8 A$ h1 V2 Y calling_format = boto.s3.connection.OrdinaryCallingFormat(),
2 P# X: U: Q" f( i )
5 e* ?! ~0 q0 y, }4 A2 L% Rbucket = conn.create_bucket("ceph-s3-bucket") / _* Z% u0 U; G4 P
for bucket in conn.get_all_buckets():
% y" t, V. s# F- l" [2 V print "{name}\t{created}".format( 5 h7 g K0 U2 O
name = bucket.name,
: `7 d! w3 L0 ?3 J+ V2 p8 T$ I3 J created = bucket.creation_date, 5 {, W2 Z O6 M l+ a: t& x9 R
)
$ a6 h) J4 n6 p) l2 b* D6 _- W6 a0 G2 ]' R2 e+ T3 u
[root@compute01 ~]# python s3client.py
1 }& v* I& a6 \+ a5 }2 h7 Kceph-s3-bucket 2022-02-09T06:35:13.546Z
# l& A1 x- {& S) a安装完rgw后，会自动创建3个pool，一个是rgw.control、rgw.meta、rgw.log，当我们创建bucket后，还会创建一个rgw.buckets.index pool池;% _0 F5 m; Q! N9 B
" i9 ^' ~ M* o4 p- Z
复制
5 [, w( Z5 p! w: A[root@compute01 ~]# ceph osd lspools
3 `& x e6 {6 N" [3 `5 images& V* J/ b7 R$ t- h
6 volumes# Z4 u, G* o. N+ {
7 backups
0 [* o6 c+ [$ `- N* q0 w" n; m8 h8 vms3 ~( x: o5 l1 C9 H0 w
9 ceph-test8 p! v+ J. v! t( k5 U$ j. h
10 .rgw.root, S1 \7 r, G" I* l& a9 M& `8 r! h$ b2 Z
11 default.rgw.control
% S. \% o; o: u0 O) X- c12 default.rgw.meta, ?5 O- K' s s& r! D8 u$ _
13 default.rgw.log& j2 d/ |2 |/ L( \& I) k
14 default.rgw.buckets.index0 A# j8 x. u% a, {8 O" m
N; H4 ~$ S0 V1 M" f6 P8 g
4 a4 [2 C- g( n6 U3. 使用命令行方式操作 rgw
x$ ?3 [7 @1 o安装命令行工具8 D1 p: @+ g0 v/ C) |$ k2 m2 H
. Y, S/ T3 [3 p* V
复制
0 b: j- s* ]) q9 f. `. h[root@compute01 ~]# yum install s3cmd -y& Q& `0 W5 |9 _
1.2 J) t6 W! s$ D' ^
配置命令行工具
0 u) \/ I2 c8 O: S! ~8 i$ Q, v9 P# L% ~ p2 `5 e! p
[root@compute01 ~]# s3cmd --configure
9 u+ v* o) v# g: e' l. u+ q- S; v
}1 \2 J! @# U' b/ i1 j# \" ~Enter new values or accept defaults in brackets with Enter.
5 d: ~) ]" D+ ~" M: uRefer to user manual for detailed description of all options.$ r. M% S! [# H$ I% n
4 l% r* b6 }3 uAccess key and Secret key are your identifiers for Amazon S3. Leave them empty for using the env variables.
6 C. P. s% k- Q& ]8 Y5 UAccess Key: 8Y28X33O2YJKI3H6B1WJ
, N! m6 Q, t- o. c6 H( R! c( ?Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y
- w$ \0 w: r6 m' Q/ R5 T0 GDefault Region [US]: # w" w7 H0 N3 ~! T
5 `5 B& Q1 l0 [. s. a. [% X! M
Use "s3.amazonaws.com" for S3 Endpoint and not modify it to the target Amazon S3., ~% R6 @' X' o2 V5 D6 B9 e
S3 Endpoint [s3.amazonaws.com]: 192.168.0.75:90004 `# A, V* n+ h. q* P: A; t
, [2 m! U& [6 D& p; d
Use "%(bucket)s.s3.amazonaws.com" to the target Amazon S3. "%(bucket)s" and "%(location)s" vars can be used# t, ~& N) g& q! T) F \3 G
if the target S3 system supports dns based buckets.
* k8 j% }: ~' W6 CDNS-style bucket+hostname:port template for accessing a bucket [%(bucket)s.s3.amazonaws.com]: 192.168.0.75:9000/%(bucket)s q k( y1 J. U
/ |3 r- c- }4 M
Encryption password is used to protect your files from reading
' O& r8 J1 q8 m. F, N" Mby unauthorized persons while in transfer to S3: b) Q2 R* G2 C% |8 o5 `; _; R
Encryption password:
4 `) M% w# ~. j4 r+ LPath to GPG program [/usr/bin/gpg]: 5 F; T4 b6 F5 x+ ^+ ?
9 j, |1 }: G3 a FWhen using secure HTTPS protocol all communication with Amazon S3
9 z5 N& h7 m' aservers is protected from 3rd party eavesdropping. This method is9 m2 Z) |3 a+ P2 h" [; B3 `$ K
slower than plain HTTP, and can only be proxied with Python 2.7 or newer7 y! w. i9 o# f6 v
Use HTTPS protocol [Yes]: no
6 v. M$ p4 Y# Y7 \9 \/ j3 e" Q; ^6 l1 g1 Q2 I" t* G# x
On some networks all internet access must go through a HTTP proxy., y- p/ {' ^2 N" V. [/ U
Try setting it here if you can't connect to S3 directly
/ ^2 N3 ~/ M7 B$ ] D6 A& GHTTP Proxy server name: ' }( X3 ~- ?7 T$ }4 Y+ ]9 Z
, z! V. A& U( H- V$ p+ P1 Z
New settings:
/ a$ C9 ?6 U5 o ]* i: J/ W' z Access Key: 8Y28X33O2YJKI3H6B1WJ
9 |) A& }: a7 K6 L8 A/ G, I8 _; \ Secret Key: ghRR17Zxs2mZrhG8DGwJqTxeTQh64wofOsGB6C5Y' ]# K/ j& \7 o9 @9 X4 V
Default Region: US
) t* e) M0 l; q, R9 W S3 Endpoint: 192.168.0.75:9000
( S' y; x* [) r+ u, q DNS-style bucket+hostname:port template for accessing a bucket: 192.168.0.75:9000/%(bucket)s
! G% I3 D2 f" Q7 T% P Encryption password: / S: L ?7 K. H- `1 s' l8 y8 x
Path to GPG program: /usr/bin/gpg2 N! E9 ?7 i0 ?# P
Use HTTPS protocol: False z+ D8 l9 p8 O* J, x. t
HTTP Proxy server name: : ^7 G# J( i6 e4 a# M9 V+ x2 p
HTTP Proxy server port: 0! N: L b) X/ [
8 T9 }' P' q9 G* DTest access with supplied credentials? [Y/n] y
! W5 p: f3 D. t9 B5 yPlease wait, attempting to list all buckets...0 x7 k7 f4 U8 p0 @
Success. Your access key and secret key worked fine :-)
& y5 n5 K: d1 R+ T8 v2 q
# c+ J( Q1 y, R- kNow verifying that encryption works...
5 F' S/ \& u. W) d' RNot configured. Never mind.2 G- F3 _( d* i9 A+ z9 D8 ?+ h
5 o' `3 h( Q/ ESave settings? [y/N] y
, `) h/ P+ S1 pConfiguration saved to '/root/.s3cfg'2 t# E( D" c$ H3 g8 j
+ r; f1 _# D4 J; Q% N& S! S2 W
4 Z6 C! q) z2 a! K/ O2 S9 J* h& a* l
7 Y& x' p7 q% O2 O, ]简单使用命令行工具3 z) I' P/ y+ F K# Y. y- S
; F* u, n4 |0 V8 K+ `[root@compute01 ~]# s3cmd ls , ~8 Q/ V9 t9 _2 b
2022-02-09 06:35 s3://ceph-s3-bucket
0 l/ {5 |, ?+ t; I
, Z" V6 r# O* f. M[root@ceph-node01 ~]# s3cmd mb s3://s3cmd-demo
0 ], U/ D( z. m i3 dERROR: S3 error: 403 (SignatureDoesNotMatch)
& d' Y0 v, R: B2 [- ^7 g) R[root@compute01 ~]# s3cmd mb s3://s3cmd-demo5 S6 K- V) z) R0 P/ r5 x
ERROR: S3 error: 403 (SignatureDoesNotMatch)
$ [( o4 k: P7 ]
- {, _3 X' t2 h6 [; [! M( x.这是需要修改版本，启用v2版本即可* L7 E5 p2 h# z
; f: n4 W( Q& C$ f# y% x# c
复制
9 u3 M- U3 g# ~0 `+ h+ @[root@compute01 ~]# sed -i '/signature_v2/s/False/True/g' /root/.s3cfg
+ u5 V* q' `" f% X9 J R3 @ q+ I, w5 U( l1 k' P
/ ?- @! e) A' s, @8 O( T再次创建
1 Q4 s9 F$ a% a7 W2 h) x5 q! o z- t7 w" h3 M* c# a
[root@compute01 ~]# s3cmd mb s3://s3cmd-demo5 @0 V3 v1 b3 t) a
Bucket 's3://s3cmd-demo/' created
]+ |* l/ a- A4 v' D
7 w9 R7 A3 {2 U) v/ f1 X. r; l. s2 M6 C& j! i
上传单个文件7 _# z" C: F n' Y8 Q! J5 e
+ h1 Q: ~& [4 ], ^* _! ~- N
- N& @3 V" ]% Q1 x, ][root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo ! m8 O* x6 s3 z3 z0 i
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
! Y* r( u5 M- E5 w 465 of 465 100% in 0s 67.71 KB/s
* O* A; k- y8 k/ c! l( E 465 of 465 100% in 0s 67.71 KB/s failed; R5 Z) T- ]0 u
WARNING: Upload failed: /fatab-demo (timed out)
3 @0 I7 A5 J9 dWARNING: Waiting 3 sec...8 w1 T( t& t$ k
upload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]
" E: E' i& w! i1 T- \& o, \# j 465 of 465 100% in 0s 361.83 KB/s
+ t4 ^# z) [' `. c4 x: m1 R# Y- j3 p/ k1 `- a1 v' s4 d1 K
出现错误 ERROR: S3 error: 416 (InvalidRange)，原因是上传object对象的时候，需要创建 pool 存储数据，创建 pool 是需要 pg，当 pg 数量不够的情况下，可以将现有的 pg 数量调小，或者修改配置文件调整参数，然后重启 mon 进程，调整方法有三种：- W; K8 Y* b0 x3 S4 P
! |$ ^) ~2 ]* x$ I6 R% U! m6 `
调整 pg_num 和 pgp_num ，默认参数值均为8;
! ^" ~* Q l) X# p调整 mon_max_pg_per_osd 参数，默认是 300，适当增大;当每个 OSD 中的 PG数量超过这个参数值时，就会报错;https://www.suse.com/support/kb/doc/?id=000019402
$ o% t( t8 p9 }/ l0 B增加更多的 OSD 进来;
1 W; v! G5 Y5 B; ~! N我们采用第2种方法：. L, [; t2 p2 k5 s9 X
" c( W- J5 Y4 X& Z[root@compute01 ~]# cat /etc/ceph/ceph.conf 6 a. t s+ M+ j0 H
[global]
1 R& H7 f$ R# B8 c" v" O7 M( Z7 ffsid = a6d0359e-8295-4e62-8620-54273462e79f
! v' F) v6 N# c& ?mon_initial_members = compute01,compute02,compute03) E- D7 K w6 U% |/ |' Q( A. Y
mon_host = 192.168.0.75,192.168.0.77,192.168.0.79
# _' l$ @- P, z9 i, O0 bauth_cluster_required = cephx
1 a7 a2 E& T5 h) c9 U$ {% v# ]7 _/ Dauth_service_required = cephx, C x1 n7 c/ R; Y
auth_client_required = cephx
7 x# z7 y2 T& t3 @ P, H" Wpublic_network = 192.168.0.0/24 - S# a4 p) w6 H! h2 O, R
cluster_network = 10.10.10.0/24
$ W; ]) @( X4 arbd_default_features = 3
, v9 `2 M! r2 i9 `5 o& Cmon_clock_drift_allowed = 2
( F: B1 G$ R* hmon_clock_drift_warn_backoff = 30" Z! q; [# [- t3 y% A; T ]. p
osd_pool_default_size = 3
9 a3 K/ j! q6 h9 Z5 v9 g9 Bosd_pool_default_min_size = 2# ~5 z) b' H5 ?7 F: w, o: N
mon_allow_pool_delete = true, S, J" P4 P& f, Q/ X
mon_max_pg_per_osd = 1000; F, w" D( }* [6 c- T
osd_crush_update_on_start = false
) B6 j4 _2 V _$ J' ]mon_pg_warn_max_object_skew = 50
( [" s4 E: q! A
0 l8 N! u+ I& ~( N# }[osd]; F( d7 D: _$ h6 H- O- W! u! G
osd_heartbeat_grace = 20
$ M* r+ ]( {2 d5 |; }# z9 tosd_heartbeat_interval = 5
2 R; C1 N5 q# V( T1 v8 j/ \0 H8 F
[client.rgw.compute01]. a) f1 W5 H9 d1 h& e2 F
host = compute01
$ z; Y. V6 ? M1 okeyring = /var/lib/ceph/radosgw/ceph-rgw.compute01/keyring
8 b8 D0 l. f! y" L. A5 V% L* n/ Nrgw frontends = "civetweb port=9000"
8 L8 j! j+ w7 _0 N6 d4 q5 C4 Orgw print continue = false
9 R& B3 b1 N! [
6 E* b, C. X# ~; Z3 r[client.rgw.compute02]- Y5 b m0 c- S: ]' R4 I" @; R
host = compute02* h$ w& p0 F/ X) l. ?1 t
rgw frontends = "civetweb port=9000"" Y% |! U! m; [% H3 C& {% {! d
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute02/keyring
+ B; X0 \: t' @7 M* ~/ T$ O6 Trgw print continue = false
. B& h6 ?6 R* ^
- d3 \: ~* G$ D, o* R; b/ ]. P. {[client.rgw.compute03]7 d/ H D3 C) G0 O$ g
host = compute03" C9 b" a& r6 s$ h+ `6 m2 j
rgw frontends = "civetweb port=9000"3 a2 H! U7 L" [
keyring = /var/lib/ceph/radosgw/ceph-rgw.compute03/keyring
5 {) L* U/ T9 [' ?rgw print continue = false* x J% |: w1 K3 S5 ~! u/ y- t
( x% m# Y% F/ @7 l9 b3 W3 n
重启下 monitor daemon 进程" p l$ p9 h- P) r/ o1 O' U k+ D
, C0 v$ V0 a# X* P
复制- V/ G/ W9 h& d6 |2 I
[root@compute01 ~]# systemctl restart ceph-mon@compute01.service & w5 p- T9 i0 V" i- `
[root@compute02 ~]# systemctl restart ceph-mon@compute02.service 2 R/ j" f, j% A0 v- C
[root@compute03 radosgw]# systemctl restart ceph-mon@compute03.service * a6 }" q$ K, S5 u1 |& H3 l
* k. X% ~# } v; i7 p# e, l
( q/ ]' m5 _+ R再次上传文件测试- l# N* ^: f& p2 C- a" m. \' J
; d$ Q; { c& a$ P4 P4 w6 v. h[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/6 k5 V4 x& }4 |5 m
upload: '/etc/fstab' -> 's3://s3cmd-demo/fstab' [1 of 1]
- v' @/ i4 q; j, `( n! N 465 of 465 100% in 0s 66.26 KB/s
4 }7 d. ?! I- O+ ?. @6 [; L- j4 o4 U$ n* b& U! U
一直卡住，应该是哪里有问题：
c& E& u3 s2 W2 w' {5 O% _; F检查过程：' K- C2 a R7 l3 y+ e
[root@compute02 ~]# ceph -s2 q' w6 I* Q; c d! G; V* a
cluster:4 |3 l1 p m& V( k
id: a6d0359e-8295-4e62-8620-54273462e79f' p3 I# l, p/ N. v" e( L
health: HEALTH_WARN% P. D# R r4 D; Y
Reduced data availability: 32 pgs inactive
" ?: y7 I; Z* T# |/ k; ]' p* h* s6 V, k2 W4 Q$ d
services:5 J, @) U0 M5 Q, F! D1 Y* L
mon: 3 daemons, quorum compute01,compute02,compute03 (age 3m)
6 H0 [: `* O/ t) k mgr: compute03(active, since 4w), standbys: compute01, compute02 c' P- ]8 m3 \
osd: 6 osds: 6 up (since 12d), 6 in (since 3w)
5 E1 z t+ ~- N9 W% M: ?5 D rgw: 3 daemons active (compute01, compute02, compute03)
( }* l5 H3 t; ~6 Q' R1 z. e3 \/ N
6 E$ V! x& h9 ~* I" A7 e( v# T6 j task status:
" u3 y) p, h& _1 S7 b
8 Z5 E7 l% P* z+ d& u; a4 A | data:
: T, I$ y: r* ~0 e2 m pools: 11 pools, 1600 pgs
; n! Z! J9 V' a7 [ objects: 65.22k objects, 258 GiB
' u( E F! ~0 u7 m) q% f9 c. _3 J8 r usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail2 e1 [8 N" K7 {+ V8 q3 b& E
pgs: 2.000% pgs unknown
% c0 H0 V6 I* W3 _ 1568 active+clean" ^8 k3 C4 @) \/ W0 D" y- r1 ?
32 unknown) W3 z& Y' Z3 x" Y+ @. n; q
1 ^' T v$ k# W8 J5 D0 r io:8 e# |* ?6 h( k( x- u1 H5 c! ]2 V
client: 114 KiB/s rd, 9.3 MiB/s wr, 144 op/s rd, 53 op/s wr
3 [$ f, P% c/ j& E: i' |7 e, v$ b$ ^: B' W {4 j& p
[root@compute02 ~]# ceph osd pool ls5 K% M$ ], E6 a$ e2 x5 e# f+ E
images' O0 B# W" V! ^- X: a6 |& [
volumes
! ]1 l7 p- Q& W+ c% Fbackups" e* {* N$ f7 E" g' W. K
vms
: p4 d0 d. k4 q& ]ceph-test
, z1 `0 a" N# M0 L.rgw.root
& E; U2 y* s" a2 q [% wdefault.rgw.control
$ d5 t/ _/ n6 [0 qdefault.rgw.meta+ ^! j( u% x9 C. ^, f
default.rgw.log Y/ @/ b" Q0 v$ d& Y" o
default.rgw.buckets.index
! J5 D- r- Z" O- ~5 m- pdefault.rgw.buckets.data3 Q# v& x# ]( r" o6 l
[root@compute02 ~]# ceph osd pool set default.rgw.buckets.data crush_rule ssd_rule2 v: ^3 E. X5 V
set pool 15 crush_rule to ssd_rule: `+ V a1 q+ U0 U4 d. q4 _
2 ~% d. C2 Z8 h& s' o! ]: G7 R
$ q7 p/ b2 T! d- R/ Q: b ^完成之后，直接显示成功了： 如下done
# `$ K+ |: T W8 e- g& O
! h& `- h% H' A; T! Y9 Z7 _[root@compute01 ~]# s3cmd put /etc/fstab s3://s3cmd-demo/fatab-demo
n7 r9 Q, n$ \2 aupload: '/etc/fstab' -> 's3://s3cmd-demo/fatab-demo' [1 of 1]7 f- {/ }3 o' a# o- \
465 of 465 100% in 0s 71.34 KB/s
1 G$ H7 K2 c1 H, u: ?4 a
8 O' T# n/ p% @, N: E% d' x
8 W; _9 K( `1 {5 Q 465 of 465 100% in 81s 5.72 B/s done
$ v* [9 k- {! V7 s1 C: C# x1 E. q4. 常用操作
' S2 M& |) P: r3 t7 X复制
9 y! L5 F. n, D( i7 u# x: A& ^# 1. 列出所有 bucket
% X5 ~4 D) I$ P6 Z2 J[root@compute01 ~]# s3cmd ls
( W, s: G+ {1 m4 h" V: [5 P7 T" |2022-02-09 06:35 s3://ceph-s3-bucket: P) b9 a3 [3 s$ X
2022-02-09 07:45 s3://s3cmd-demo$ A1 l+ f: ~( Z( i j" t& ^
' S3 l( U4 q7 {# 2. 创建 bucket $ J' b" [% F& C+ W2 w
[root@compute01 ~]# s3cmd mb s3://gwj-demo/
2 ]( t) H8 X* Q1 ~8 m2 y2 EBucket 's3://gwj-demo/' created
" u3 \4 y, W2 q# f" T/ f3 V1 ~2 z( }% W) G
[root@compute01 ~]# s3cmd ls
* A. I- y L4 {: B$ [2 w+ P2022-02-09 06:35 s3://ceph-s3-bucket
3 {' E7 ^( T# i" `* r2022-02-09 08:01 s3://gwj-demo1 i$ d/ ]; { v6 X, O
2022-02-09 07:45 s3://s3cmd-demo3 e3 _5 a6 S+ _1 w% v
1 u6 R1 o% \; G' \! j. J+ |
; ^7 N& O6 B, C- ~! e7 [# 3. 删除空的bucket
! E0 h5 j e- L% M8 I[root@compute01 ~]# s3cmd rb s3://gwj-demo, ?- l; ^" M+ I! e
Bucket 's3://gwj-demo/' removed
9 t/ V8 R" u# L+ T
" ]4 ?# ~; Z7 B$ E2 I& O[root@compute01 ~]# s3cmd ls, x; L" S* e- Z. E% {8 E, A9 @; Z
2022-02-09 06:35 s3://ceph-s3-bucket8 c9 _$ x/ {1 s$ j2 R
2022-02-09 07:45 s3://s3cmd-demo
- q H* g" O: A1 ]4 @+ C( M; F3 P
4 ^' a- o; c5 p! m- O! V% |# 4. 上传文件到bucket
) b5 X' t u- \: ?# K: |[root@compute01 ~]# s3cmd put ip s3://s3cmd-demo
& \# g" j5 l* G5 P% }/ `upload: 'ip' -> 's3://s3cmd-demo/ip' [1 of 1]
# M* `/ p \# q' k 5 of 5 100% in 0s 296.75 B/s done4 p7 o9 l' b8 ?+ ]% L6 N4 [3 A! v0 h
! N* R6 u8 l7 O1 A& |
# 5. 上传目录到 bucket 9 V0 R7 |& o2 Y2 j9 k/ X9 X
[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/2 w( `. a: ~/ M/ j+ a' [
ERROR: Parameter problem: Use --recursive to upload a directory: ./
" v$ |) K3 K4 H$ m2 u# Z. ~9 L" a
4 r8 N+ w. E [[root@compute01 ~]# s3cmd put ./ s3://s3cmd-demo/ --recursive
. L7 M! U3 f9 e, |upload: './.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638530173.61-8631-41487900105782/AnsiballZ_command.py' [1 of 177]$ k6 h0 k, M: M2 y- d- k Y/ y
106754 of 106754 100% in 0s 2.29 MB/s done" B( A8 S$ d' i I2 E* ~: Q
upload: './.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539832.68-4173-41041061132542/AnsiballZ_copy.py' [2 of 177]
# L: K6 t6 B& H5 g' j4 o v/ | 111789 of 111789 100% in 0s 2.13 MB/s done
, D5 R3 O9 B9 d2 o7 S+ [0 Kupload: './.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' -> 's3://s3cmd-demo/.ansible/tmp/ansible-tmp-1638539833.48-4173-36283133816628/AnsiballZ_copy.py' [3 of 177]
$ J+ v1 ~. T/ r7 M, L0 V. G 111789 of 111789 100% in 0s 1485.51 KB/s done8 w: r/ {. k. T7 g8 `
4 z$ S V; n2 f6 ?
9 S5 }$ O2 ?7 R6 i; p) X
又卡住了：
8 r6 E: n! K( D( j, @) b检查：; Q. n s1 b% d0 }( P7 a1 U9 Q
[root@compute01 ~]# ceph -s
. T7 j, ^% h% m, ~3 }2 T/ M cluster:
% H3 h2 L& e6 w s id: a6d0359e-8295-4e62-8620-54273462e79f
% Y& ~3 a4 K' ]1 u health: HEALTH_WARN; l7 D, J3 ? Q G, _) t) e& ~" \0 C
Reduced data availability: 32 pgs inactive( V7 W3 J+ G0 S4 `' G E2 u0 J
, @$ u1 V4 R& e. J services:% i" A+ b/ A7 e; ~, ]& b: Q
mon: 3 daemons, quorum compute01,compute02,compute03 (age 15m)
9 A" J4 l* ]3 o* N: ]4 X: a$ f9 a mgr: compute03(active, since 4w), standbys: compute01, compute02
9 a+ U* Y9 e4 O0 E; S. v osd: 6 osds: 6 up (since 12d), 6 in (since 3w)
# `. { `0 C+ X2 f7 H rgw: 3 daemons active (compute01, compute02, compute03)
8 M& ?0 x( J6 I& y
! g+ C9 R9 F: d& A/ W% S/ @, N# z7 g task status:6 r0 |9 T; j" Y ]. S3 \
( q. A/ P5 b# a( t1 [% C) B data:# @% Q# S. t2 N' H8 Y( h _$ @
pools: 12 pools, 1632 pgs
. D' ?3 T7 P! G9 Z: j objects: 65.40k objects, 258 GiB
6 Y& r/ U! W( F2 w* x% d2 z1 D usage: 754 GiB used, 4.5 TiB / 5.2 TiB avail
, z( K" E& }! _; {1 W pgs: 1.961% pgs unknown
; | d) `% ?/ y' S 1600 active+clean/ T1 r3 m/ a( |" ^$ h
32 unknown
4 d) C1 W& P4 P% H$ Z
, V* L' [* C: P: b io:! u8 i9 k0 Y8 F
client: 117 KiB/s rd, 11 MiB/s wr, 145 op/s rd, 50 op/s wr, `$ z1 i& L7 W/ |! D; C
0 _0 G& @( `) f) Q% @7 d& z[root@compute01 ~]# ceph osd pool ls
7 V* L, c$ P& a$ }, P; ?2 Oimages
, h+ B' T0 l: nvolumes" @" @* p1 l! H) M; k
backups
) ^8 o2 F; h9 m# I( J$ h' D; K/ }* rvms
2 i' n) E8 l. d: f; s3 ?ceph-test+ U' z# K' `/ W4 O; B" R: d
.rgw.root0 `+ R% L" _# i- B
default.rgw.control) m3 j% B. Q6 v& l
default.rgw.meta
, B& m; F' m. [8 V$ I8 B8 idefault.rgw.log3 f, |' V& Q# e. [( L, h$ m
default.rgw.buckets.index) ]( n/ F+ r2 R4 X5 s8 L
default.rgw.buckets.data
c T; B9 `8 \8 q0 k9 X+ }$ Q R7 n: Kdefault.rgw.buckets.non-ec
$ b) T8 W* @& g+ F1 T[root@compute01 ~]# ceph osd pool set default.rgw.buckets.non-ec crush_rule ssd_rule
9 P- r3 |* P! B+ v! ?$ E' Iset pool 16 crush_rule to ssd_rule
% v$ R1 w" [7 C0 a0 w! V
/ S9 u9 j4 A9 f' X5 m
; H$ N3 B, o! M# |设置完成，上传完成：9 C- s* n+ K3 G6 ^+ r3 R. \, g& ^' C
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 1 of 6, 15MB] [174 of 177]
K8 t, r9 N7 B) R 15728640 of 15728640 100% in 2s 5.38 MB/s done! G1 I- d; A% S* B/ a
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 2 of 6, 15MB] [174 of 177]: U6 J) D, d: q- W. j4 L
15728640 of 15728640 100% in 0s 38.25 MB/s done f* e5 m4 Q8 l- |, {" r8 x
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 3 of 6, 15MB] [174 of 177]& Q+ R8 C4 D6 l0 l5 p. S7 f6 [4 T- V
15728640 of 15728640 100% in 0s 22.08 MB/s done5 t$ o4 j7 F, Q+ @
upload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 4 of 6, 15MB] [174 of 177]5 `8 B+ H) r2 c2 r
15728640 of 15728640 100% in 0s 15.15 MB/s done
+ c- K/ U) [" B) @9 R! d6 T- W5 hupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 5 of 6, 15MB] [174 of 177]
' n5 }" {) z- m; _" \ 15728640 of 15728640 100% in 1s 11.45 MB/s done
/ U$ a7 u! L* J3 U3 r9 U' j& gupload: './monitor/telegraf' -> 's3://s3cmd-demo/monitor/telegraf' [part 6 of 6, 11MB] [174 of 177]
/ j9 T& \! ]9 E& O& u! e. G* v3 u1 q 11599872 of 11599872 100% in 0s 21.31 MB/s done3 Z- b* c: m% w' A7 @
upload: './monitor/telegraf_system.conf' -> 's3://s3cmd-demo/monitor/telegraf_system.conf' [175 of 177]) \- E3 u5 b3 W1 l& [7 z: @. [
4322 of 4322 100% in 0s 87.82 KB/s done& E8 c+ L1 S* n6 |' R
upload: './osds.map' -> 's3://s3cmd-demo/osds.map' [176 of 177]. K0 ^) H. R) g3 o
5895 of 5895 100% in 0s 120.84 KB/s done5 c5 e5 g; o1 F/ E9 c, A- I* A
upload: './s3client.py' -> 's3://s3cmd-demo/s3client.py' [177 of 177], {- u. x8 m: ^+ B* @! @& q m- |' u
659 of 659 100% in 0s 13.74 KB/s done" i1 h4 C7 E6 |- `, B; F: |
8 u4 R! z9 c5 P, r6 N2 u4 |$ _
9 g8 `! D/ x& ], L h8 P5 R1 g
4 {& ?$ \. N' Q1 b" Y# 6. 列举 bucket 中的内容 5 ~/ p" W( | w6 _2 ~, o2 M
[root@compute01 ~]# s3cmd ls s3://s3cmd-demo/4 i* J6 B. z* D2 q
DIR s3://s3cmd-demo/.ansible/
- i' X( l% a" z9 q. Q DIR s3://s3cmd-demo/.cache/$ v! q9 X1 y- v _/ J: D
DIR s3://s3cmd-demo/.gnupg/
0 [* U3 p7 o& R, Z8 L DIR s3://s3cmd-demo/.ssh/
- @. U& K# T$ V. N1 } DIR s3://s3cmd-demo/monitor/5 b, Z5 w9 v0 k4 @2 A1 z3 Y
2022-02-09 08:08 25196 s3://s3cmd-demo/.bash_history
: j! c2 \& ~1 C% j) [2022-02-09 08:08 18 s3://s3cmd-demo/.bash_logout
6 u! I! X; w' ?8 q2 n+ r* N* ^9 k2022-02-09 08:08 176 s3://s3cmd-demo/.bash_profile1 O1 G2 f' `) x' s
2022-02-09 08:08 176 s3://s3cmd-demo/.bashrc
. g3 }( f7 l1 d) T9 |( Y. Z# N2022-02-09 08:08 1077 s3://s3cmd-demo/.cephdeploy.conf
# L0 F: m0 v* R4 n J2022-02-09 08:08 100 s3://s3cmd-demo/.cshrc5 M3 E! _& |! _, V
2022-02-09 08:08 42 s3://s3cmd-demo/.lesshst
. Z3 ^, i% l3 B0 I2 N* |2022-02-09 08:08 2248 s3://s3cmd-demo/.s3cfg0 m1 F1 ^7 o+ }- M% C5 B+ T
2022-02-09 08:08 129 s3://s3cmd-demo/.tcshrc& w. n( p) V' c, b6 M. F
2022-02-09 08:08 6490 s3://s3cmd-demo/.viminfo1 G; X% p) {0 J: r3 r9 L6 u
2022-02-09 08:08 34064 s3://s3cmd-demo/1111. ]) Y8 ~) @* R
2022-02-09 08:08 1797 s3://s3cmd-demo/anaconda-ks.cfg! j/ n, }; N$ ?) m
2022-02-09 08:08 86604 s3://s3cmd-demo/ceph-grafana-dashboards-14.2.8-111.el7cp.noarch.rpm# `8 b0 x0 z7 W/ y" P
2022-02-09 08:08 9625568 s3://s3cmd-demo/ceph-mgr-dashboard-14.2.8-111.el7cp.noarch.rpm4 c: u( F4 ^8 ^+ b+ \9 l0 Q* s
2022-02-09 07:52 465 s3://s3cmd-demo/fatab-demo2 ?7 z* J# L( L
2022-02-09 07:58 465 s3://s3cmd-demo/fstab
; u) E: f; {# F) V5 W* n. a2022-02-09 08:08 5 s3://s3cmd-demo/ip
- L" r5 y, u; j4 `+ G2022-02-09 08:11 5895 s3://s3cmd-demo/osds.map
3 z3 N% S1 Z) {; n$ y2022-02-09 08:11 659 s3://s3cmd-demo/s3client.py. f/ X5 P2 k8 U; z9 { }3 `( |% y
+ Z: q# }+ |- D. f6 ]! N! M! \
# 7. 下载单个文件 1 `- a" f+ y3 v
[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip
. h) ?# M7 x" X+ O, e7 Mdownload: 's3://s3cmd-demo/ip' -> './ip' [1 of 1]* ?" i* I" ^' d: Z' E
5 of 5 100% in 0s 396.17 B/s done
- n; [* O& ?! C, n- q
# C. x0 x/ r; w3 k[root@compute01 tmp]# ls( `7 ]- V2 w0 n
ip
2 o4 o& E0 E+ i- V! ~5 C( W% a1 ?5 J5 G/ i1 Q& |
# 8. 删除 bucket 中的内容
6 x( z: t+ s3 {" Z5 s }[root@compute01 tmp]# s3cmd del s3://s3cmd-demo/ip/ l. F1 _+ N ^; n4 ?/ A5 `( J
delete: 's3://s3cmd-demo/ip'4 I+ d# o' f- v/ {
) Y9 z. _. }+ B2 E
[root@compute01 tmp]# s3cmd get s3://s3cmd-demo/ip+ B% g. h0 c1 P: X- W* {) d7 ~2 E
ERROR: Parameter problem: File ./ip already exists. Use either of --force / --continue / --skip-existing or give it a new name.
! h/ q& e7 P+ p6 v- o4 c[root@compute01 tmp]# ! h( C: A3 A S
$ j$ o1 ^' q0 Z# I+ E$ D
# 9. 获取对应的bucket所占用的空间大小 8 u$ Q( y" ^$ u p5 w7 x5 u
[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/+ z, P0 M% s9 h$ y% I2 s6 y
104M 178 objects s3://s3cmd-demo/
6 v8 K- z" B, C* o3 j9 n. k[root@compute01 tmp]# s3cmd du -H s3://s3cmd-demo/.ssh! a4 u( i! E+ s9 W, m+ p# Q
4K 4 objects s3://s3cmd-demo/.ssh
! ~2 x" Z0 T# d$ l# z; [3 a, e
4 o; F8 c" M n1 d7 g: \; z0 J# 10. 查看bucket文件信息
! b) I6 k, I" q4 R* z! y& O# J/ m4 Q[root@compute01 tmp]# s3cmd info s3://s3cmd-demo/fstab
' x: P4 n! {" Z5 C# l! xs3://s3cmd-demo/fstab (object):
$ F5 H$ k& o' p0 Z File size: 465# f5 q4 n0 ~: v* _* c/ @. z/ ?' T4 i, l
Last mod: Wed, 09 Feb 2022 07:58:45 GMT( F( G9 Q* A; q' M
MIME type: text/plain
8 F3 D# Q- ~+ x: } Storage: STANDARD% S& p H4 T) f7 m) S3 {
MD5 sum: b3b65116db83a743d6e80ae6c31896ef
) z' H% t1 {; s+ U$ T( h SSE: none. b7 K+ `: j; q0 r) P
Policy: none9 Z0 h" Q/ u8 { \, G" h5 R
CORS: none1 ]( k% t( D% {6 ^' I( G; Z. W
ACL: Ceph S3 User Demo: FULL_CONTROL
5 m, w& H! t+ Y x-amz-meta-s3cmd-attrs: atime:1644387423/ctime:1638501772/gid:0/gname:root/md5:b3b65116db83a743d6e80ae6c31896ef/mode:33188/mtime:1638501362/uid:0/uname:root
) [' g8 |. Q1 |# G6 T% ^. F* b5 Y) c# J" X
# 11. 两个bucket之间相互cp
% \3 e# S {0 |, T[root@compute01 gwj]# s3cmd cp s3://s3cmd-demo/ip s3://test-demo/
; \7 I& R' ]- w- \1 j+ s2 k! wremote copy: 's3://s3cmd-demo/ip' -> 's3://test-demo/ip' , b3 T R# h; c: L5 K/ V
[root@compute01 gwj]# s3cmd cp --recursive s3://s3cmd-demo/.ssh s3://test-demo/ 4 P. D7 U# m1 l8 w2 v2 h$ l
remote copy: 's3://s3cmd-demo/.ssh/authorized_keys' -> 's3://test-demo/.ssh/authorized_keys' 8 A: p1 w2 B1 ~
remote copy: 's3://s3cmd-demo/.ssh/id_rsa' -> 's3://test-demo/.ssh/id_rsa' " G( z7 t- R+ } N6 \; d
remote copy: 's3://s3cmd-demo/.ssh/id_rsa.pub' -> 's3://test-demo/.ssh/id_rsa.pub'
6 C" @/ g& g( V& rremote copy: 's3://s3cmd-demo/.ssh/known_hosts' -> 's3://test-demo/.ssh/known_hosts' $ p9 u! | T# W5 N. r
[root@compute01 gwj]#
1 F. \/ H: h' m ^" P0 v; l: o7 a6 Y: k) f2 Z3 H# n
# 12. 两个bucket之间进行mv操作 " z* s) B# o X- g& T$ u/ C: f
[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp 7 l1 t' }% ?5 N
2020-10-06 10:24 12288 s3://s3cmd-demo/.swp
9 V1 X* i8 x5 I[root@compute01 gwj]# s3cmd mv s3://s3cmd-demo/.swp s3://test-demo/
: m$ O* t! s& S0 jmove: 's3://s3cmd-demo/.swp' -> 's3://test-demo/.swp'
; m# v# {" F3 S$ y- E: Y0 X[root@compute01 gwj]# s3cmd ls s3://test-demo/.swp - n- T3 L$ B+ P
2020-10-06 10:36 12288 s3://test-demo/.swp
5 a8 H. [) g0 E v, Y& Z[root@compute01 gwj]# s3cmd ls s3://s3cmd-demo/.swp * { f* [# p. `! D' i
[root@compute01 gwj]#
6 ]; I, H: W) F, j5 V7 r! m9 N! c# q7 y" |& B' Z& r* n4 R3 g" } m( L
# 13. 列出需要同步的文件和目录，但不进行同步
& V% \4 R- D6 ]! s" u4 ~" s7 g[root@compute01 ~]# s3cmd sync --dry-run ./ s3://s3cmd-demo
1 m( M3 ]7 c: U9 W- n! F' cupload: './.swp' -> 's3://s3cmd-demo/.swp'
' {, Q7 [/ a: J8 e1 m, x) dupload: './ip' -> 's3://s3cmd-demo/ip' & ~- s! A- A9 l, k# A
upload: './.cache/abrt/lastnotification' -> 's3://s3cmd-demo/.cache/abrt/lastnotification'
; e' ^) M, g8 ~, ]7 fremote copy: 'size.log' -> 'gwj/size.log' 1 |/ s/ \+ D4 A& [! l; C
WARNING: Exiting now because of --dry-run
% L5 l* w) c" l3 v[root@compute01 ~]# / p# F. B5 ~! C. C. M- m, W9 R: V( U
% G; {4 B* O4 g/ C, _
# 14. 在bucket中删除本地不存在的文件
$ a" Z# `: y. n[root@compute01 a]# ls
' K2 l- X2 T ]; W- \# T10.txt 1.txt 2.txt 3.txt 4.txt 5.txt 6.txt 7.txt 8.txt 9.txt
2 J, v0 G2 @3 _, x6 {[root@ceph-node01 a]#
/ ?! n. c N$ m) i[root@ceph-node01 a]# s3cmd ls s3://test2-demo/
( B& T. y( i v3 j2 ~7 K1 S( d+ k
( d. k" `0 T" ^0 T[root@compute01 a]# rm -rf 10.txt 2 Z$ i( P! Y! V, J: B
[root@compute01 a]# s3cmd sync --delete-removed ./ s3://test2-demo/
. c5 }; i9 \9 K/ e: hdelete: 's3://test2-demo/10.txt' 8 \5 ^) D; ~6 v6 w6 N9 t
[root@ceph-node01 a]# s3cmd ls s3://test2-demo/
. R% ^4 N( I8 \ S+ {9 B+ C" O1 B
! x& k7 z' y( r[root@compute01 a]# ) y/ Y( W1 U5 V) ]) n+ h
* V* m A# E5 X3 n
" j$ l9 C4 w* H/ W, b
使用 swift 访问 RGW
: }, I7 M; \( h. z$ K) e0 ^1. 创建 swift 用户
' U5 O" M6 N) @/ t复制
# e5 X5 H) G% l, B[root@compute01 ceph-deploy]# radosgw-admin subuser create --uid ceph-s3-user --subuser=ceph-s3-user:swift --access=full
( [$ Q4 o/ r& B2 j& a{ & k7 {3 T3 V( t
"user_id": "ceph-s3-user", 7 b$ o9 o/ M# p( j Z
"display_name": "Ceph S3 User Demo",
/ s4 X; W x2 v. \0 j "email": "", " `2 s" @/ j. z- _
"suspended": 0, 6 m+ C7 x8 u+ x2 G7 ]6 M3 y9 W8 D
"max_buckets": 1000, - c& C. t9 ]6 G! y4 Q' \
"subusers": [
- `) r5 A; J" y* t' w { / x# `, |7 X- Y2 r* m9 i
"id": "ceph-s3-user:swift", 7 E8 X, k, j9 c: Q' \* f% H! p% e
"permissions": "full-control"
8 J& n" t' A7 O0 J* [8 u) q2 Y( ~ }
4 T. V8 B S( @: Z1 U' N ],
8 ?2 M1 [$ Z1 S, H1 f "keys": [
, o+ H s, k* w8 o ~! v' N) A { 1 }5 `) K3 {- Y! u, h
"user": "ceph-s3-user", ' s8 h6 D& ^# x$ B, f% c
"access_key": "V3J9L4M1WKV5O5ECAKPU", * {5 {5 H3 c# R$ H4 u% Z5 s8 L. ?
"secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw"
3 n! o/ n$ f$ i* r* i- R0 w } 4 L- [4 {' i) j0 g
],
. e6 b+ @: z+ `7 A& d- n "swift_keys": [
9 ^5 H4 y: v$ l9 Y. u { ) j8 F/ H7 g$ P6 I. O9 F' M
"user": "ceph-s3-user:swift", " z. o8 k2 X& C+ ?
"secret_key": "ZIOOU8Xcfe3m6ZZapK5P2rU0GGPaiS31chy9yvMW"
) M+ b3 J( s' f) R# ^1 {- ]0 N5 r. L }
- P! j6 ]4 T: W( T! m+ K) x# k ],
# Z0 V; T) B* V5 m: u1 f "caps": [], , c" L! y1 M- S5 N5 M4 }
"op_mask": "read, write, delete", 9 \; d( A& m: T4 E7 p
"default_placement": "", # n6 | E. Z& J! ?$ S6 u a; R1 P
"default_storage_class": "", , d0 W: r2 T" \! ]1 P
"placement_tags": [], ) c, M7 @" q9 N$ w
"bucket_quota": { ' \4 a$ g3 q/ q5 |5 {
"enabled": false,
: t- q, C9 w6 e; U8 q% W' i "check_on_raw": false,
1 R D8 w7 f4 B9 f5 v# t "max_size": -1, / N5 ]* m; k3 R5 B
"max_size_kb": 0,
) }+ V5 x2 T9 Z "max_objects": -1
6 r4 w% X" R* `- x7 h8 F% h! ]. a% y }, , J% d c, f8 N# S! X0 T
"user_quota": { ! K4 C5 N1 b1 W* l8 x8 }4 e
"enabled": false, 7 P4 I6 `& c6 P# D. z+ I
"check_on_raw": false, ; Z- A' k9 p( h* O; [, R
"max_size": -1,
" @0 G; l+ p' z" l% Z "max_size_kb": 0, $ e: P) g/ j1 n s, S# K
"max_objects": -1
. k& G4 g/ }9 x+ @, Q },
7 I; j& z: M, W "temp_url_keys": [],
! Q& I0 b. \' O, Q7 w2 q "type": "rgw",
0 ?* z5 L8 `4 q6 c! v "mfa_ids": []
2 s! t# ]0 ^3 T% G! F} + u! A; a: U: W& z0 D
1 B% m4 E9 w. }3 D* {
[root@compute01 ceph-deploy]# " t: K/ S$ X+ x- i: x5 k) i
* Y- b. B% U6 p2 q- t, ~
% ]" K9 m2 M# p2. 创建 swift 用户的secret* }8 Y( ?( k! q2 @3 t
复制8 u; r9 p6 J) }0 |6 u3 n, _
[root@compute01 ceph-deploy]# radosgw-admin key create --subuser=ceph-s3-user:swift --key-type=swift --gen-secret 7 m* v5 ]1 W" G1 J% b- m' C; e
{ : Y+ U1 C" A' p; H
"user_id": "ceph-s3-user", % M' R1 A. P' w; |2 j
"display_name": "Ceph S3 User Demo", + a& [. Y) }, W+ I! h, p
"email": "", # ~# J7 N. G1 Q3 l0 O' o
"suspended": 0, ( Y& L) s# D/ o4 w" E$ U7 _
"max_buckets": 1000,
/ ^4 ^! n7 P" ~( G, v "subusers": [
+ a4 M9 b, n" e" M; p: y" {7 u' Z { $ \1 \% s. X `9 W" u0 Z5 Q
"id": "ceph-s3-user:swift",
$ `5 c6 L4 M1 _- C "permissions": "full-control" 6 m3 Y* C, U3 _" X. J# o" C7 d
} & f* [% T# Y5 e+ z& m/ ]9 Y# v+ W5 Z
],
2 c+ I- }# b+ t1 Z. f0 h "keys": [ ( v2 _, e' \. [5 U* P4 g( E1 N
{
7 G+ Z% ]# k' Y' E3 I* N "user": "ceph-s3-user",
2 p; a8 r' j4 v! i2 D "access_key": "V3J9L4M1WKV5O5ECAKPU",
# s9 C& R. E% G0 W2 l: E! b "secret_key": "f5LqLVYOVNu38cuQwi0jXC2ZTboCSJDmdvB8oeYw"
2 u( ]- G: {2 f: A3 j6 i( D } ' x; R5 Q( A" k* y% X& S1 |
], : w& c" h/ w) d u1 c, S
"swift_keys": [ * |& u S) \, b$ s+ \+ p k1 p
{ # t) `, o8 ?. e& N8 l
"user": "ceph-s3-user:swift",
$ g* {- t2 [. i% ~( R% j "secret_key": "0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE" 5 O8 y( Y, D/ J, e5 n; F
}
( X* \' y% t' {( i# ^" Q ],
6 p, ?( p2 j) O "caps": [],
8 n% \- E: X6 M9 ]" F "op_mask": "read, write, delete",
( P: F9 o0 B3 A; A" ^# X& C "default_placement": "",
0 q4 q( m' s2 H9 n "default_storage_class": "",
, b1 h8 W5 A u( q8 G& p' x9 L "placement_tags": [], % [, }/ w4 B( w+ l
"bucket_quota": { 7 z2 E2 h) L! [) K) ^
"enabled": false,
: z- M+ @! O% E$ P5 K, U+ J, W& E "check_on_raw": false,
! _ {# \) {( P9 \2 g. q8 Q+ [ "max_size": -1, : l$ G% B9 i- z$ N% o! C2 z
"max_size_kb": 0,
: y7 d, Z) W' d# X6 c5 Y7 g "max_objects": -1 0 }; m) Z% \2 q6 r; Y* t% I7 q8 G
},
' D7 ]+ e( W2 D! U3 `7 r "user_quota": {
4 B/ ]- u; m* L8 H/ J# B# a "enabled": false,
/ p# h$ G1 I( @# J% ] "check_on_raw": false,
. G" p. E& T; t/ z; l! {; u5 d "max_size": -1,
, }$ \" z3 l2 [3 t8 [/ G "max_size_kb": 0, / J x1 B G2 V' p+ H* K6 I8 V
"max_objects": -1
% G3 t1 M; w2 `2 C- \2 c },
$ m2 F, {0 {+ ~& s) R "temp_url_keys": [], ( X5 K3 L2 Y5 |' p3 {
"type": "rgw",
0 U, G/ W9 K, k; A2 r. P- ^ "mfa_ids": [] ; R. s' t/ W2 E! S3 @* l/ I0 Q. {
} 1 b& V, Y7 W" i) F4 \4 i) g, Z5 }
, Y( P, f4 H6 ], {[root@ceph-node01 ceph-deploy]# . C$ l6 x8 `7 l ?$ Q% u4 R
) x# A c4 C8 s6 q: C/ j3 ?2 i( a, O. o# J A
3. 需要使用 pip 安装 swift 客户端工具
. }/ I8 U7 w% A( g6 r. \, B2 P复制2 l, F/ R$ I# U( j6 b( o
# 注意如果有pip的话，就不需要再安装了 4 ~2 B' }. i; o( x- O1 p
[root@compute01 ceph-deploy]# curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
5 A. X7 a& @4 s5 U+ K。。。 / @; R' [4 V" [/ q8 Z4 [& j% W' j
[root@compute01 ceph-deploy]# python get-pip.py 7 j9 j+ I* j1 [+ |+ }, u
。。。
" O6 d# ?$ h4 h% U9 X1 j8 f+ `[root@compute01 ceph-deploy]# pip install python-swiftclient 9 v- B2 A8 h1 g/ e: |
。。。
! [0 y: H+ U* q" ]" ^6 Z, L2 Y( h6 d$ v
8 Q9 q6 }1 D$ Z3 o' h& a2 p- w4. 使用 swift 命令行工具
' l7 A o! V5 |# `0 W- z- u复制
- c) a$ u+ ]8 ~# ~, e3 K[root@compute01 ceph-deploy]# swift -A http://100.73.18.152/auth -U ceph-s3-user:swift -K 0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE list & p9 K+ I F) n6 ? u9 e
ceph-s3-bucket 4 c! j% _1 Y0 i- [" s
s3cmd-demo
: m( ] |8 V$ Z" `. C/ p+ T[root@compute01 ceph-deploy]#
- N, {6 W% X- \8 m v$ s+ {) `: N' g4 }8 r O$ g7 J
9 k3 w" V9 X. C. Y+ } t! `5. 配置成环境变量的形式 o/ N) o' H+ ?! A# K( C) W
复制
" ^$ A) S% t4 Z; n[root@compute01 ceph-deploy]# cat /etc/profile ; v" z! C, W, Q9 ]& E
。。。 ; M3 b; b- j3 k+ Q) u! g
export ST_AUTH=http://100.73.18.152/auth
, v; s6 n: J4 F( Q* l( a3 r. zexport ST_USER=ceph-s3-user:swift
|3 e' c+ k" _% R; B2 Jexport ST_KEY=0M1GdRTvMSU3fToOxEVXrBjItKLBKtu8xhn3DcEE
& v s2 @5 {% @9 q- m1 A8 a[root@ceph-node01 ceph-deploy]# source /etc/profile 1 N5 A! J [# N* H
[root@ceph-node01 ceph-deploy]# swift list
% Z& y& [% ? @9 @' K4 g) Tceph-s3-bucket ( w$ @) ~9 \/ O N* W
s3cmd-demo 2 k4 j/ ^) x. E0 K8 f
[root@compute01 ceph-deploy]# 5 Q, J6 W; N1 y. L6 E/ e7 m3 v8 `
3 W. v5 J& G' T* s d6 ?, x/ E' i4 v [# o- P
6. 创建bucket6 L+ j. y/ P1 g9 R* g3 b* b. u
复制) o4 _, s' h1 f# J" y$ e) P
[root@compute01 ceph-deploy]# swift post swift-demo 4 U, d6 d# _: P) Q" ~
[root@compute01 ceph-deploy]# swift list
Y1 P; d9 U6 G$ ]4 lceph-s3-bucket 9 ]) b( X; Q- a
s3cmd-demo ) B3 ]6 n4 d" u: R
swift-demo 6 Q+ w- m: _+ M P! k# n# x Z+ [
[root@compute01 ceph-deploy]# $ M" p1 T" S5 t8 ]! T* ]9 l k9 j
" j$ c# l5 \9 j
* y+ I! N" @( ?( B# S7. 上传单个文件测试5 E0 F0 i j* y+ O/ T
复制$ }3 u5 i' R% r' O9 U B4 G5 h1 J4 k
[root@compute01 ceph-deploy]# swift upload swift-demo /etc/fstab
4 k- }! ? q" r. r* Q* UObject HEAD failed: http://100.73.18.152/swift/v1/swift-demo/etc/fstab 416 Requested Range Not Satisfiable
) `# @! h- A/ H[root@compute01 ceph-deploy]#
; c' z2 u/ V; b, \* y8 U
6 ]4 b, u$ g& V a7 P8. 上传目录测试
1 ^4 |( z0 }* t( H& M复制: Q% H% `: v4 K8 u7 |
[root@compute01 a]# swift upload swift-demo /etc/fstab
; d+ J$ u7 w: x6 q6 s7 e5 }& ?4 Fetc/fstab % B/ e- a- v6 O0 h1 Y* G
[root@compute01 a]# ! ]. C) F; O6 E" Z& a5 K
0 C0 {! T9 w3 `; @5 }8 R* w
9. 常用操作
, N# _3 ^9 T! q% x) O- t: O/ H复制
6 i* f+ M3 Z3 E! e) `1 C# 1. 列举所有 bucket
" `+ @ E. G8 `7 D! n" h* \. n7 c[root@compute01 a]# swift list
+ H) R+ }3 x$ b5 lceph-s3-bucket 8 l( T2 [$ M+ j; [- j( D
s3cmd-demo : Y5 n' J) c# N+ b6 f# X# b
swift-demo
' }- ~- A+ v2 `( C8 Ttest-demo ' B3 y/ {6 ^( ^4 s* `) b; H, ^
test2-demo
0 B+ ]4 u) {9 V9 v/ p, w[root@compute01 a]# ( G( `/ s5 `. d7 L
; M2 b3 s# [+ Z/ m: b1 _4 T( I
# 2. 列举所有 bucket & H g5 e- @& R1 j7 C5 ~* w& e
[root@compute01 a]# swift list --lh + P4 G! U5 q% T, ^
0 0 2020-10-06 04:13:10 ceph-s3-bucket
: w, P; j8 }- U* W$ |& h! I P 37 3.6M 2020-10-06 04:34:49 s3cmd-demo
- q4 F3 y: d4 x' ~ 2360 33M 2020-10-06 08:07:55 swift-demo ! D" [ J. i* B/ `3 n R
7 16K 2020-10-06 10:32:02 test-demo / G# C x3 I& [
9 387 2020-10-06 11:17:00 test2-demo
" W) o! ]8 n l* t$ t 2.4K 36M / {! w# k5 I& J# j9 e9 G9 L, Z
[root@compute01 a]#
3 [, E% R+ \" L
) [ v" Y) L: [# 3. 列举单个 bucket 0 _1 E0 W& g' j3 C: G1 ^
[root@compute01 a]# swift list swift-demo
. p7 Y: h4 D) e$ d1 R6 W. ?' S3 p( y0 E
# 4. 上传单个文件到bucket % |3 ^& W; | o& ?8 M; O: r$ J
[root@compute01 a]# swift upload swift-demo /etc/fstab
/ h8 i& n' o- _0 Aetc/fstab
- V, G! F; W P8 V5 f9 N[root@compute01 a]# {( `2 V' G) ?5 G' u
( O7 c: v4 T' Z' h4 c
# 5. 上传目录到指定的bucket / r# G5 M( x v I9 n
[root@compute01 a]# swift upload swift-demo /etc/ 0 i/ ^! b' k% N$ L& j6 v1 V4 R
% K3 d _+ \$ v7 N
# 6. swift 状态信息 , V# l! ]' o8 t" h
[root@compute01 a]# swift stat
! ~2 A3 Q8 G, a8 o$ `9 q Account: v1
( w5 C: V" P! f0 q6 C* d C, u Containers: 5
; M, y; w" n" r2 y3 ` Objects: 2413
; `# p! T! U: u( K! z Bytes: 38701415 * N3 \) M, V; V, T
Objects in policy "default-placement-bytes": 0 8 E; A8 M* P! Q
Bytes in policy "default-placement-bytes": 0
0 x9 n9 x$ s- J4 C Containers in policy "default-placement": 5
+ Z% {; y! N0 n8 A Objects in policy "default-placement": 2413
N; U3 S$ C1 h# P' Q7 y3 R% U& t5 c Bytes in policy "default-placement": 38701415 , F; I" T6 j% X: E3 X) T9 }# z
X-Openstack-Request-Id: tx000000000000000001302-005f7c5afd-a638-default 2 s' F: A4 N" n+ `# Q6 j
X-Account-Bytes-Used-Actual: 45948928
, z4 x: m4 }$ R8 r) Z, W% F' y8 N4 X X-Trans-Id: tx000000000000000001302-005f7c5afd-a638-default
5 a5 H4 O4 f8 D. Z" `$ i" t- Y5 m X-Timestamp: 1601985277.38095 4 z3 w" B. U+ D/ q% P' a, J2 F
Content-Type: text/plain; charset=utf-8 , Z( `$ \# {4 f2 O
Accept-Ranges: bytes
. o, d( X( Z- ` u* @; U- n[root@compute01 a]# / f% b* y$ E( X" h/ P* w5 }! Q) Y
4 `& ^: |* h, H/ i+ a) }& ^) b# 7. 创建 bucket
0 s$ L! a7 T" f[root@compute01 a]# swift post swift-test 1 _! ?" u" F H' ?4 e: m
[root@compute01 a]# swift list
% [$ E$ _* T( {! I- ^ceph-s3-bucket
8 x4 _) M+ y7 O3 n is3cmd-demo
9 [1 D9 _/ K" W1 @. t5 zswift-demo
& H: Z2 \9 d U0 oswift-test ) T- z4 @& a# e: c) e( A; U) W- {! Z
test-demo
& ]5 x2 W0 k3 u& r0 N+ Ztest2-demo
4 y# E1 d; X9 [9 K7 ][root@compute01 a]# 3 T, D0 A' `3 t- B+ K7 Z
- x% o- a# U% i! t
# 8. 删除 bucket / X" q5 I# T( F) @' v
[root@compute01 a]# swift delete swift-demo
) ^7 W( B. J# [8 {/ k9 W8 n- h* e* D! [6 L
# 9. 删除指定 object ' p1 o* ^# e$ ?: J5 B
[root@compute01 a]# swift delete swift-test root/a/1.txt " C8 p- U* m% P5 v* j
root/a/1.txt
, ]2 Q4 k/ l0 u% r$ P[root@compute01 a]# . [+ k6 u) `: k g+ Z: m' A
+ q# j# C0 ?% T! q1 T
# 10. 上传大文件时可以使用-S指定分片大小
, o7 e; _; f3 C2 a+ m' `1 `& h[root@compute01 a]# swift upload swift-test /home/log.txt
) Z7 ~/ H+ s0 |home/log.txt
" \5 V3 w! s$ ^$ d3 }- r! e6 W[root@compute01 a]# swift upload swift-test -S 102400000 /home/log2.txt
5 ?) H# H0 T# z# K$ thome/log2.txt segment 5
( m% ?' Y N0 }3 b6 B0 L; Z2 dhome/log2.txt segment 3 - |1 i2 Q# b: S' r
home/log2.txt segment 1
9 I. U( t( y) O+ X- O' R) _home/log2.txt segment 0 ) N8 \8 s: e* i) x
home/log2.txt segment 2
8 R; @' Q0 M& h) Q8 Vhome/log2.txt segment 4 - D- D% q' q& {5 t
home/log2.txt , B6 e& O1 l! J) e# r7 w4 o Y
[root@ceph-node01 a]# " N3 g: l! V0 Y- H) M2 I/ n
z9 S9 M5 b7 U/ c总结
; q2 E, i$ y$ g9 u从概念、安装、到实践使用，简单介绍了对象存储命令行工具。
$ F) U3 x0 o9 U: d' J- e) j. N% r
, P, i2 ?% s% _, Q: H4 h8 |. Y
1 J& O+ N8 s% N5 F0 s) J |
|
发表于 2022-2-9 15:15:20