Our RHCS overview already explained the basics of RHCS. This time we will take two hosts with shared storage and provide KVM guests as services.

Installation of the nodes

• /tmp/ and /var/ should be running on different partitions, this improves performance.
• Activate Debian backports, especially for the Kernel.
• Make sure all IP addresses can be resolved in both directions - /etc/hosts helps here in worst case.
• The host name must not resolve to 127.0.0.1! You would only get problems with the Cluster Management System CMAN.
• /etc/hosts/ and /etc/resolv.conf should be the same on all nodes.
• Create password free ssh keys for all nodes and distribute them.
• For ultimate performance it is best to install the latest Debian Linux kernel. In our example we used linux-image-2.6.32-bpo.2-amd64, which crashes the guest kernels >= 2.6.30. However, a patch is available, see bug #573071.
• The network devices should be named in a way that makes sense, for example: rhcs-backbone and external instead of eth0 and eth1.

Configuring the shared storage

apt-get install iscsitarget iscsitarget-source 
echo 'ISCSITARGET_ENABLE=true' > /etc/default/iscsitarget
m-a a-i iscsitarget

Keep in mind that the iSCSI target must build properly, see bug #566740. The configuration of the shared storage is done via /etc/ietd.conf:

IncomingUser discovery_in YourSecurePwd1
OutgoingUser discovery_out YourSecurePwd2
Target YOURMACHINE:clvm1
       IncomingUser node_in YourSecurePwd1
       OutgoingUser node_out YourSecurePwd2
       Lun 0 Path=/dev/sdx1,Type=blockio

On the nodes the same target must be accessed, so make sure /etc/iscsi/iscsid.conf is correct:

discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = discovery_in
discovery.sendtargets.auth.password = YourSecurePwd1
discovery.sendtargets.auth.username_in = discovery_out
discovery.sendtargets.auth.password_in = YourSecurePwd2
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = node_in
node.session.auth.password = YourSecurePwd1
node.session.auth.username_in = node_out
node.session.auth.password_in = YourSecurePwd2

The service is started with /etc/init.d/open-iscsi start. Existing targets can be searched, deleted or added by the following commands:

# discovering the targets
iscsiadm -m discovery -t st -p YOURMACHINE -P 1
# deleting target on wrong interface
iscsiadm -m node -p 192.168.0.100:3260,1 -o delete
# opening the portal
iscsiadm -m node --targetname "iqn.2010-03.YOURMACHINE:clvm1" --portal "YOURMACHINE:3260" --

VM setup

apt-get install linux-image-2.6.32-bpo.2-amd64 kvm libvirt-bin virtinst -t lenny-backports

When configuring the bridge, make sure that the bridge name is the same on all nodes. Also the libvirt configuration must be the same on all hosts, so it makes sense to use puppet or similar techniques.
Afterwards, bring up the guests with:

virt-install -n <NAME> -r 256 --vcpus=1 --disk path=/dev/vg_cluster#/<LV> \
  -c /root/debian-<VERSION>-amd64-netinst.iso --vnc --noautoconsole --os-type linux \
  --os-variant debianLenny --accelerate --network=bridge:bridge0 --hvm -k de

To monitor the process use virt-viewer -c qemu+ssh://:/system .

RHCS setup

invoke-rc.d nfs-kernel-server stop
invoke-rc.d nfs-common stop
invoke-rc.d portmap stop
update-rc.d -f nfs-kernel-server remove
update-rc.d -f nfs-common remove
update-rc.d -f portmap remove

Btw., system-config-cluster is not available for Lenny, but our Philipp Hübner has created a backport:

wget --no-check-certificate https://www.credativ.com/~phu/lenny-backports/system-config-cluster/system-config-cluster_1.0.53-1_all.deb
dpkg -i system-config-cluster_1.0.53-1_all.deb
apt-get -f install
apt-get install xauth

In order to have locking on the LVM cluster, you now need to modify /etc/lvm/lvm.conf: check for the global part.

 locking_type = 3

With the newer kernels the module lock_dlm also vanished, so CMAN init script must be modified: comment out the line modprobe lock_dlm 2>&1 || return 1. Additionally, RHCS 2 only supports XEN, so for libvirt you need to load the resource handler vm.sh.

wget --no-check-certificate https:///www.credativ.com/~phu/vm.sh -O /usr/share/cluster/vm.sh
chmod +x /usr/share/cluster/vm.sh

RHCS itself is called via

/etc/init.d/cman start
/etc/init.d/clvm start
/etc/init.d/rgmanager start

Fencing

Closing words

Introduction

• Each configuration change is done only once, regardless of the actual number of maintained servers. Unnecessary - and pretty boring - repetition is avoided, lucky us!
• The configuration is streamlined for all machines, which makes it much easier to maintain.
• A central infrastructure makes it easier to quickly get an overview about the setup - "running around" is not necessary anymore.
• Last but not least, a central configuration tree enables you to incorporate a simple version control of your configuration: for example, playing back the configuration "PRE-UPDATE" on all machines of an entire setup only takes a couple of commands!

Technical workflow

• The master can hand over files which the node copies to a defined place - if it does not already exist.
• The node is asked to check certain file and directory permissions and to correct them if necessary.
• Depending upon the operating system, the node checks the state of services and starts or stops them. It can also check for installed packages and if they are up to date.
• The master can force the node to execute arbitrary commands

Installation

Puppet, main configuration

[modules]
allow 192.168.0.1/24
allow *.credativ.de

Configuration of the configuration - Modules

The SSH module

1. The package open-ssh must be installed and be the newest version.
2. Each node's sshd_config file has to be the same as the one saved on the master.
3. In the event that the sshd_config is changed on any node, the sshd service should be restarted.
4. The user credativ needs to have certain files in his/her directory $HOME/.ssh.

mkdir -p /etc/puppet/modules/ssh/manifests
mkdir -p /etc/puppet/modules/ssh/files

class ssh{
        package { "openssh-server":
                 ensure => latest,
        }
        file { "/etc/ssh/sshd_config":
                owner   => root,
                group   => root,
                mode    => 644,
                source  => "puppet:///ssh/sshd_config",
        }
        service { ssh:
                ensure          => running,
                hasrestart      => true,
                subscribe       => File["/etc/ssh/sshd_config"],
        }
        file { "/home/credativ/.ssh":
                path    => "/home/credativ/.ssh",
                owner   => "credativ",
                group   => "credativ",
                mode    => 600,
                recurse => true,
                source  => "puppet:///ssh/ssh",
                ensure  => [directory, present],
        }
}

package

Here we make sure that the package openssh-server is installed in the newest version.

file

A file on the node is compared with the version on the server and overwritten if necessary. Also, the rights are adjusted.

service

Well, as the name says, this deals with services: in our case the service sshd must be running on the node. Also, in case the file /etc/ssh/sshd_config is modified, the service is restarted automatically.

file

Here we have again the file type, but this time we do not compare a file, but an entire directory.

Nodes and modules

import "ssh"

node default {
    include rsyslog
}
node 'external' inherits default {
    include ssh
}

Certificates - secured communication between nodes and master

Closing words

In such cases you can still rebuild the old package with a modified rpm file: download the source RPM, install it with
rpm -Uvh dovecot-1.0.7-7.el5.src.rpm
get the diff from below and apply it to the spec file:
patch < dovecot.diff
Download the sources as given in the now modified spec file to your SOURCES directory, and rebuild the package:
rpmbuild -ba dovecot.spec
and welcome the new sieve plugin dovecot-sieve-1.0.4-7.x86_64.rpm. Install it and continue as usual. And as a small help for writing Sieve scripts: you can verify them on various online services like the one from the PHP Sieve library.

Be careful, however: you have to maintain this package on your own - especially when a dovecot update comes along or when the sieve plugin code is updated. Do bear in mind, though, that this information, as with all howtos, should be followed at your own discretion; it comes with no warranty, and might eat your cats.

And here is the patch for the spec file:

--- dovecot.old.spec    2010-03-11 09:59:38.598277799 +0100
+++ dovecot.spec        2010-03-11 09:58:08.639526842 +0100
@@ -1,7 +1,10 @@
 %define upstream 1.0.7
+%define sieve_upstream 1.0.4
 %define pkg_version 1.0.7
 %define my_release 7
 %define pkg_release %{my_release}%{?dist}
+%define pkg_sieve_version 1.0.4
+%define pkg_sieve_release %{my_release}%{?dist}
 
 Summary: Dovecot Secure imap server
 Name: dovecot

@@ -12,6 +15,7 @@
 
 %define build_postgres 1
 %define build_mysql 1
+%define sieve_name dovecot-sieve
 
 Source: http://dovecot.org/releases/%{name}-%{upstream}.tar.gz
 Source1: dovecot.init

@@ -22,6 +26,7 @@
 Source6: perfect_maildir.pl
 Source7: dovecot-REDHAT-FAQ.txt
 Source8: dovecot.sysconfig
+Source9: http://dovecot.org/releases/sieve/%{sieve_name}-%{sieve_upstream}.tar.gz
 Patch100: dovecot-1.0.7-default-settings.patch
 Patch102: dovecot-1.0.rc2-pam-setcred.patch
 Patch103: dovecot-1.0.beta2-mkcert-permissions.patch

@@ -80,6 +85,16 @@
 primarily in mind.  It also contains a small POP3 server.  It supports mail 
 in either of maildir or mbox formats.
 
+%package sieve
+Requires: %{name}
+Summary: CMU Sieve plugin for dovecot LDA
+Group: System Environment/Daemons
+Version: %{pkg_sieve_version}
+Release: %{pkg_sieve_release}
+
+%description sieve
+This package provides the CMU Sieve plugin for dovecot LDA.
+
 %prep
 %setup -q -n %{name}-%{upstream}

@@ -94,6 +109,8 @@
 %patch503 -p1 -b .CVE-2008-4577
 %patch504 -p1 -b .CVE-2008-4870
                               
+%setup -q -n %{name}-%{upstream} -D -T -a 9
+
 %build
 rm -f ./configure
 libtoolize -f

@@ -115,6 +132,16 @@
 
 make %{?_smp_mflags}
 
+cd %{sieve_name}-%{sieve_upstream}
+rm -f ./configure
+libtoolize -f
+autoreconf
+%configure                           \
+    INSTALL_DATA="install -c -p -m644" \
+    --with-dovecot=../
+
+make %{?_smp_mflags}
+
 %install
 rm -rf $RPM_BUILD_ROOT
 make install DESTDIR=$RPM_BUILD_ROOT

@@ -169,6 +196,11 @@
 mv $RPM_BUILD_ROOT%{docdir} $RPM_BUILD_ROOT%{docdir}-%{version}
 mkdir -p $RPM_BUILD_ROOT/var/lib/dovecot
 
+# dovecot-sieve
+pushd %{sieve_name}-%{sieve_upstream}
+make install DESTDIR=$RPM_BUILD_ROOT
+popd
+
 %pre
 /usr/sbin/useradd -c "dovecot" -u %{dovecot_uid} -s /sbin/nologin -r -d /usr/libexec/dovecot dovecot 2>/dev/null || :
 
@@ -243,6 +275,9 @@
 %attr(0750,root,dovecot) %{docdir}-%{version}/examples/mkcert.sh
 %attr(0750,dovecot,dovecot) %dir /var/lib/dovecot
 
+%files sieve
+%defattr(-,root,root)
+%{_libdir}/%{name}/lda/lib90_cmusieve_plugin.so
 
 %changelog
 * Mon Nov 24 2008 Michal Hlavinka <mhlavink@redhat.com> - 1.0.7-7