Step 1:
yum install drbd83.x86_64 kmod-drbd83.x86_64 heartbeat.x86_64
Step 2:
#vim /etc/hosts
127.0.0.1 localhost.localdomain localhost
127.0.1.1 localhost.localdomain localhost
10.20.0.100 master.your.domain master
10.20.0.101 node01.your.domain node01
10.20.0.102 node02.your.domain node02
192.168.0.101 drbd-node01 ha-node01
192.168.0.102 drbd-node02 ha-node02
Step 3: prepare hdd’s with
#fdisk /dev/cciss/c0d0 ← this is for a HP disk array, your disk may be different, perhaps /dev/sda
The number of cylinders for this disk is set to 53132.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-53132, default 1058): ← enter for default
Using default value 1058
Last cylinder or +size or +sizeM or +sizeK (1-53132, default 53132): ← enter for default
Using default value 53132
Command (m for help):t
Partition number (1-4): 1
Hex code (type L to list codes): 83 ← hex code for linux system
Command (m for help):w
The partition table has been altered!
Calling ioctl() to re-read partition table.
/dev/cciss/c0d0p1 is now partitioned.
#####################
#Using LVM (if not using LVM skip to DRBD.conf Setup):
#####################
fdisk /dev/cciss/c0d0 this is for a HP disk array, your disk may be different, perhaps /dev/sda
The number of cylinders for this disk is set to 53132.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-53132, default 1058): ← enter for default
Using default value 1058
Last cylinder or +size or +sizeM or +sizeK (1-53132, default 53132): ← enter for default
Using default value 53132
Command (m for help):t
Partition number (1-4): 1
Hex code (type L to list codes): 8e ← hex code for lvm
Command (m for help):w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Now we have a blank partition set to a Linux LVM type. Now we need to setup LVM on it:
[root@node1 ~]# sudo pvcreate /dev/cciss/c0d0p1 # create a physical volume on /dev/cciss/c0d0p1Now we have a 100G logical volume for our DRBD setup.
Physical volume "/dev/cciss/c0d0p1" successfully created
[root@node1 ~]# sudo vgcreate -s 16M vg0 /dev/ccis/c0d0p1 #create a volume group named vg0 using /dev/cciss/c0d0p1 physical volume
Volume group "vg0" successfully created
[root@node1 ~]# sudo lvcreate -L 100G -n lvol0 vg0 #create a logical volume of 100G named "lvol0" in vg0
Logical volume "lvol0" created
Step 4:
[root@node1 ~]# vim /etc/drbd.conf
global {
usage-count no;
}
common {
protocol C;
handlers {
pri-on-incon-degr "echo 'DRBD: primary requested but inconsistent!' wall; /etc/init.d/heartbeat stop";
"halt -f";
pri-lost-after-sb "echo 'DRBD: primary requested but lost!' wall; /etc/init.d/heartbeat stop";
"halt -f";
}
syncer {
rate 10M;
al-extents 1801;
}
startup {
wfc-timeout 0;
degr-wfc-timeout 15;
}
disk {
on-io-error detach;
# fencing resource-and-stonith;
}
net {
sndbuf-size 64K;
timeout 60; # 6 seconds (unit = 0.1 seconds)
connect-int 10; # 10 seconds (unit = 1 second)
ping-int 10; # 10 seconds (unit = 1 second)
ping-timeout 5; # 500 ms (unit = 0.1 seconds)
max-buffers 2048;
max-epoch-size 2048;
cram-hmac-alg "sha1";
shared-secret "secret"; # should change this to
}
}
resource r0 {
on ha-node01 {
disk /dev/vg0/lvol0; # location of lvm volume, use /dev/cciss/c0d0p1 for non-lvm
address 192.168.111.125:7788;
device /dev/drbd0; # name of drbd device
meta-disk internal;
}
on ha-node02 {
disk /dev/vg0/lvol0; # location of lvm volume, use /dev/cciss/c0d0p1 for non-lvm
address 192.168.111.126:7788;
device /dev/drbd0;
meta-disk internal;
}
}
[expand title=”Show Descriprion”]
The global section is allowed only once, the most important option is usage-count. Usage-count contacts the DRBD http server every time a new DRBD version is installed on the system for statistics purposes. Set it to no.
The common section sets options inherited to every resource.
Protocol C is the replication mode of DRBD we are using and is the “safest”. There are currently three modes of DRBD replication:
1.Protocol A: Asynchronous replication. Local writes on primary are complete as soon as the disk write has occurred and the replication packet has been sent to the local TCP send buffer. If a forced-failover occurs, the most recent update may be lost. (http://www.drbd.org/users-guide/s-replication-protocols.html)
2.Protocol B: Memory synchronous replication. Local writes on primary are complete as soon as the disk write has occurred and the replication packet has reached the remote node. If simultaneous power failure occurs on both nodes the most recent writes will be lost. (http://www.drbd.org/users-guide/s-replication-protocols.html)
3.Protocol C: Synchronous replication. Local writes on primary are complete only once both the local write and the remote disk write are confirmed. Data loss will occur if both disk subsystems are irreversibly destroyed at the same time. (http://www.drbd.org/users-guide/s-replication-protocols.html)
The two handlers I have first are used with heartbeat and tell heartbeat what to do if the DRBD is in an inconsistent state or has lost the primary role. They are unnecessary for DRBD to function, but useful in a heartbeat setup.
The syncer rate is the maximum bandwidth a resource will use for resynchronization in the background . It does not make sense to make this rate higher than the max write rate on the secondary node disk I/O, nor the
bandwidth available on the network. We can not push more data than can be written or transmitted. Good rule of thumb according to drbd.org is use 30% of your available replication rate, so take the bottleneck and use 30% of it, i.e. 110MB/s effective bandwidth X 0.3 = 33 MB/s, so 33M would be your syncer rate (rate is in bytes not bits per second). (http://www.drbd.org/users-guide/s-configure-syncer-rate.html)
Startup is setting some timeouts, wfc-timeout is the wait for connection timeout, you can limit the amount of time DRBD waits for the rest of the cluster to connect up; degr-wfc-timeout is used if this is the only node in a cluster and the cluster is rebooted. These timeouts are in seconds.
Disk is the section where you can configure lower-level I/O error handling. The on-io-error sets the strategy to one of three options. Detach, recommended option, the node will drop the backing device and continue in a diskless mode. You must replace the disk or manually re-attach the device to continue using this node. Pass_on, DRBD will pass the error onto the upper disk handling layer. On the primary DRBD reports the error to the mounted file system, on the secondary this is ignored since there is no mounted file system. Call-local-io-error, will invoke a command or script designated by the administrator.
The net section sets up some network settings, most of which are self-explanatory.
The resource section defines your resources by name, here we have defined resource r0. We are using disk /dev/vg0/lvol0 (which is our lvm volume), we define the address and port for the server and we are using an internal meta-disk. An internal meta-disk is nice since it moves around with the disk, but is not always optimal, due to more disk activity, but do have the advantage of easier disk replacement upon failure since the meta data is part of the diskset. Otherwise you can define an external meta-disk by giving its location, i.e. meta-disk /dev/sda7[0];
[/expand]
Formatting Disk and Meta-Disk
Now we have DRBD configured. We need to initialize the meta data and format the disk still. DRBD itself is not a disk format, but a lower level mechanism to achieve a network based RAID-1.
On both hosts create the Meta-disk staging for resource r0:
sudo drbdadm create-md r0
or
drbdsetup /dev/drbd0 primary -o
Now on Host1 tell DRBD that this is the primary (Do this only on host1):
sudo drbdadm primary r0
Format the disk with ext3 on Host1 only, drbd will automatically format the disk on host2, whether you are setting up both hosts at the same time or not, drbd will synchronize the disk upon attachment.
sudo mkfs.ext3 -b 4096 /dev/drbd0
You are done, Check to see if all is well:
[root@node1 ~]cat /proc/drbd or sudo /etc/init.d/drbd statusThis shows that the current computer is Primary, we have a Connected Secondary and both are UpToDate. If it states Standalone instead of Connected, we cannot contact the secondary. This command shows us some important information pertaining to DRBD and this hosts status.
version: 8.3.0 (api:88/proto:86-89)
GIT-hash: 9ba8b93e24d842f0dd3fb1f9b90e8348ddb95829 build by root@host1, 2009-01-29 09:46:57
1: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r---
ns:18819524 nr:0 dw:18819524 dr:3163845 al:828 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0
To view what each statistic refers to go to http://www.drbd.org/users-guide-emb/ch-admin.html
What next:
You can mount your drbd partition on the primary and propagate data to it.
[root@node1 ~]sudo mnt -t ext3 /dev/drbd0 /mnt
Having Heartbeat mount and umount your DRBD partition as needed (optional):
So now you have a DRBD partition and have the data propagated to it. Let’s set up heartbeat to utilize this partition and only mount the device if we are the primary node in the HA cluster:
[root@nodeX ~]sudo vi /etc/ha.d/haresources
host1 IPaddr::192.168.0.100/24/eth0 drbddisk::r0 Filesystem::/dev/drbd0::/mnt::ext3 apache
this is a typical haresources file that takes over the 192.168.0.100 address, takes over the primary role on DRBD r0, mounts /dev/drbd0 to /mnt with the ext3 filesystem and starts up apache. The only DRBD specific parts fo this haresources file is drbddisk::r0 Filesystem::/dev/drbd0::/mnt::ext3.