我的书签
添加书签
移除书签实验环境
由于实验环境中并没有ssd磁盘，所以这里操作时假设每一个主机有一块ssd盘，操作时手动把对应的osd调整class标签。
修改crush class
1、查看当前osd分布情况
ID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF-1 0.74797 root default-15 0.21999 rack rack1-9 0.08800 host node13 hdd 0.04399 osd.3 up 1.00000 1.0000010 hdd 0.04399 osd.10 up 0.98001 1.0000011 hdd 0.04399 osd.11 up 0.96002 1.00000-3 0.13199 host storage-04 hdd 0.04399 osd.4 up 0.96002 1.000005 hdd 0.04399 osd.5 up 0.96002 1.0000014 hdd 0.04399 osd.14 up 0.98001 1.00000-16 0.26399 rack rack2-5 0.13199 host node20 hdd 0.04399 osd.0 up 0.98628 1.000006 hdd 0.04399 osd.6 up 1.00000 1.0000016 hdd 0.04399 osd.16 up 1.00000 1.00000-7 0.13199 host storage-12 hdd 0.04399 osd.2 up 1.00000 1.000008 hdd 0.04399 osd.8 up 1.00000 1.0000012 hdd 0.04399 osd.12 up 1.00000 1.00000-17 0.26399 rack rack3-11 0.13199 host node31 hdd 0.04399 osd.1 up 1.00000 1.000007 hdd 0.04399 osd.7 up 1.00000 1.0000015 hdd 0.04399 osd.15 up 1.00000 1.00000-13 0.13199 host storage-29 hdd 0.04399 osd.9 up 1.00000 1.0000013 hdd 0.04399 osd.13 up 1.00000 1.0000017 hdd 0.04399 osd.17 up 0.96002 1.00000
2、查看当前集群的crush class
# ceph osd crush class ls["hdd",]
3、删除osd.0，osd.1，osd.2，osd.3，osd.4，osd.9的class
# for i in 0 1 2 3 4 9 ; do ceph osd crush rm-device-class osd.$i ; donedone removing class of osd(s): 0done removing class of osd(s): 1done removing class of osd(s): 2done removing class of osd(s): 3done removing class of osd(s): 9# ceph osd treeID CLASS WEIGHT TYPE NAME STATUS REWEIGHT PRI-AFF-1 0.74797 root default-15 0.21999 rack rack1-9 0.08800 host node13 0.04399 osd.3 up 1.00000 1.0000010 hdd 0.04399 osd.10 up 0.98001 1.0000011 hdd 0.04399 osd.11 up 0.96002 1.00000-3 0.13199 host storage-04 0.04399 osd.4 up 0.96002 1.000005 hdd 0.04399 osd.5 up 0.96002 1.0000014 hdd 0.04399 osd.14 up 0.98001 1.00000-16 0.26399 rack rack2-5 0.13199 host node20 0.04399 osd.0 up 0.98628 1.000006 hdd 0.04399 osd.6 up 1.00000 1.0000016 hdd 0.04399 osd.16 up 1.00000 1.00000-7 0.13199 host storage-12 0.04399 osd.2 up 1.00000 1.000008 hdd 0.04399 osd.8 up 1.00000 1.0000012 hdd 0.04399 osd.12 up 1.00000 1.00000-17 0.26399 rack rack3-11 0.13199 host node31 0.04399 osd.1 up 1.00000 1.000007 hdd 0.04399 osd.7 up 1.00000 1.0000015 hdd 0.04399 osd.15 up 1.00000 1.00000-13 0.13199 host storage-29 0.04399 osd.9 up 1.00000 1.0000013 hdd 0.04399 osd.13 up 1.00000 1.0000017 hdd 0.04399 osd.17 up 0.96002 1.00000
4、设置osd.0，osd.1，osd.2，osd.3，osd.4，osd.9的class为ssd
5、再次查看crush class，多了个ssd
# ceph osd crush class ls["hdd","ssd"]
# ceph osd crush rule create-replicated rule-ssd default rack ssd# ceph osd crush rule dump rule-ssd"rule_id": 1,"rule_name": "rule-ssd","ruleset": 1,"type": 1,"min_size": 1,"max_size": 10,"steps": [{"op": "take","item": -30,"item_name": "default~ssd"},{"op": "chooseleaf_firstn","num": 0,"type": "rack"},{"op": "emit"}]}
验证
方法一：
1、获取crushmap
# ceph osd getcrushmap -o monmap60
2、反编译crushmap
可以看到在crushmap中多了一项rule-ssd的crush rule
3、测试
# crushtool -i monmap --test --min-x 0 --max-x 9 --num-rep 3 --ruleset 1 --show_mappingsCRUSH rule 1 x 0 [3,2,9]CRUSH rule 1 x 1 [2,4,9]CRUSH rule 1 x 2 [1,4,0]CRUSH rule 1 x 3 [9,0,3]CRUSH rule 1 x 4 [2,9,3]CRUSH rule 1 x 5 [1,2,4]CRUSH rule 1 x 6 [1,3,0]CRUSH rule 1 x 7 [1,0,4]CRUSH rule 1 x 8 [0,4,1]CRUSH rule 1 x 9 [0,1,3]
方法二：
1、创建名为ssdtest的pool，并设置crush rule为rule-ssd
# ceph osd pool create ssdtest 64 64 rule-ssdpool 'ssdtest' created[root@node1 ~]# ceph osd pool get ssdtest crush_rule
2、上传对象
# rados -p ssdtest put init.txt init.sh
3、查询对象的OSD组
从查询结果可以看出，对象所在的OSD，是class为ssd的OSD
