Preivous: Installing Ceph Next: Install KVM, Orchestrate worker nodes
Listing images/blocks in a pool and removing
root@server1:~# ceph osd lspools 1 .mgr 2 rbdpool root@server1:~# rbd ls rbdpool testblockdevice root@server1:~# rbd rm rbdpool/testblockdevice Removing image: 100% complete...done. root@server1:~# ceph osd pool rm rbdpool rbdpool --yes-i-really-really-mean-it pool 'rbdpool' removed root@server1:~#
Removing the monitor node (from any existing monitor node)
NODETOREMOVE=storage4
OTHMON1=storage2
OTHMON2=storage3
ceph mon remove $NODETOREMOVE
monmaptool /etc/ceph/monmap --rm $NODETOREMOVE
scp /etc/ceph/monmap $OTHMON1:/etc/ceph/monmap
scp /etc/ceph/monmap $OTHMON2:/etc/ceph/monmap
ssh $OTHMON1 "chown ceph:ceph /etc/ceph/monmap"
ssh $OTHMON1 "systemctl stop ceph-mon.target"
ssh $OTHMON1 "systemctl start ceph-mon.target"
ssh $OTHMON2 "chown ceph:ceph /etc/ceph/monmap"
ssh $OTHMON2 "systemctl stop ceph-mon.target"
ssh $OTHMON2 "systemctl start ceph-mon.target"
Removing the manager node (from any server)
NODETOREMOVE=storage4
ceph mgr fail $NODENAME
ssh $NODETOREMOVE "systemctl stop ceph-mgr.target"
ssh $NODETOREMOVE "systemctl disable ceph-mgr.target"
Changing replication factor
ceph osd pool set nvmepool size 2
Mounting block device, unmounting and deleting image
rbd create --size 10G --pool ssdpool ssdbd rbd map ssdbd --pool ssdpool mkfs.ext4 /dev/rbd0 mkdir /root/test mount /dev/rbd0 /root/test rbd unmap ssdpool/ssdbd rbd rm ssdpool/ssdbd
Get crush map
ceph osd getcrushmap -o crushmap.bin
crushtool -d crushmap.bin -o crushmap.txt
Set crush map
crushtool -c crushmap.txt -o crushmap-new.bin
ceph osd setcrushmap -i crushmap-new.bin
Removing OSD from the cluster
- Identify the OSD (ceph osd tree)
- Mark the OSD as ‘out’ (ceph osd out <osd_id>)
- Wait for Rebalancing to Complete (ceph -w)
- Remove the OSD from the CRUSH Map (ceph osd crush remove <osd_id>) – May not be required
- Stop the OSD Service (systemctl stop ceph-osd@<osd_id>.service)
- Purge the OSD (ceph ods purge <osd_id> –force
- Remove OSD Data, lvdisplay, lvremove, vgdisplay, vgremove, pvdisplay pvremove, remove the partition using fdisk