May 2010 Archives

The PostgreSQL developers' community recently published the first Beta version of the new 9.0 release. Over 200 new functions and improvements feature in this new version.

With this new release, PostgreSQL now amongst other features claims an inbuilt replication solution as well as the ability to access and read standby nodes, continuously being updated by Log Shipping (Hot Standby). Streaming replication allows the sending of transaction logs directly to one or more standby nodes, which considerably reduces the amount of time lost compared with the more common, file-based log shipping. Combining these two features makes for an extremely efficient solution for high availability or loadbalanced systems.

The all new PostgreSQL version also offers the following innovations:

• Memory based LISTEN/NOTIFY: this replaces the previous table based implementation and is much faster.
• Exclusion Constraints: broadens constraints to be able to deal with the complex datatypes of overlapping constraints.


• Procedural code such as PL/pgSQL, PL/Perl and PL/Python can now be done inline per DO command.
This means there is no longer need to define a function with CREATE FUNCTION.
• Triggers on columns


• Triggers can now be tied to conditions


• Named argument lists for procedures


• Parameters can now be flexibly linked to rolls/databases

As always, anyone interested is invited to share their test results with the developers. Information on the procedure for testing and filing of error messages can be found in the Wiki.

All blog articles which fall into the PostgreSQL category are grouped in their own feed, and if you find you need support and services for PostgreSQL, you've come to the right place at credativ.

Following our earlier introduction to RHCS we now present a real world example: the installation of RHCS with Debian to provide certain virtual machines as services.

Our RHCS overview already explained the basics of RHCS. This time we will take two hosts with shared storage and provide KVM guests as services.

Installation of the nodes

In this setup the nodes are the machines which are running KVM. Each running KVM guest is a service managed by RHCS. While installing the KVM hosts you should make sure you comply with the following suggestions:

• /tmp/ and /var/ should be running on different partitions, this improves performance.
• Activate Debian backports, especially for the Kernel.
• Make sure all IP addresses can be resolved in both directions - /etc/hosts helps here in worst case.
• The host name must not resolve to 127.0.0.1! You would only get problems with the Cluster Management System CMAN.
• /etc/hosts/ and /etc/resolv.conf should be the same on all nodes.
• Create password free ssh keys for all nodes and distribute them.
• For ultimate performance it is best to install the latest Debian Linux kernel. In our example we used linux-image-2.6.32-bpo.2-amd64, which crashes the guest kernels >= 2.6.30. However, a patch is available, see bug #573071.
• The network devices should be named in a way that makes sense, for example: rhcs-backbone and external instead of eth0 and eth1.

Configuring the shared storage

As with almost any HA solution, a key element of RHCS is the shared storage which is accessed by all the nodes. In this example we take a "private" machine and install an iSCSI target on it:

apt-get install iscsitarget iscsitarget-source 
echo 'ISCSITARGET_ENABLE=true' > /etc/default/iscsitarget
m-a a-i iscsitarget

Keep in mind that the iSCSI target must build properly, see bug #566740. The configuration of the shared storage is done via /etc/ietd.conf:

IncomingUser discovery_in YourSecurePwd1
OutgoingUser discovery_out YourSecurePwd2
Target YOURMACHINE:clvm1
       IncomingUser node_in YourSecurePwd1
       OutgoingUser node_out YourSecurePwd2
       Lun 0 Path=/dev/sdx1,Type=blockio

On the nodes the same target must be accessed, so make sure /etc/iscsi/iscsid.conf is correct:

discovery.sendtargets.auth.authmethod = CHAP
discovery.sendtargets.auth.username = discovery_in
discovery.sendtargets.auth.password = YourSecurePwd1
discovery.sendtargets.auth.username_in = discovery_out
discovery.sendtargets.auth.password_in = YourSecurePwd2
node.startup = automatic
node.session.auth.authmethod = CHAP
node.session.auth.username = node_in
node.session.auth.password = YourSecurePwd1
node.session.auth.username_in = node_out
node.session.auth.password_in = YourSecurePwd2

The service is started with /etc/init.d/open-iscsi start. Existing targets can be searched, deleted or added by the following commands:

# discovering the targets
iscsiadm -m discovery -t st -p YOURMACHINE -P 1
# deleting target on wrong interface
iscsiadm -m node -p 192.168.0.100:3260,1 -o delete
# opening the portal
iscsiadm -m node --targetname "iqn.2010-03.YOURMACHINE:clvm1" --portal "YOURMACHINE:3260" --

VM setup

The virtual machines are provided by KVM. Thus the apropriate KVM software must be installed first:

apt-get install linux-image-2.6.32-bpo.2-amd64 kvm libvirt-bin virtinst -t lenny-backports

When configuring the bridge, make sure that the bridge name is the same on all nodes. Also the libvirt configuration must be the same on all hosts, so it makes sense to use puppet or similar techniques.
Afterwards, bring up the guests with:

virt-install -n <NAME> -r 256 --vcpus=1 --disk path=/dev/vg_cluster#/<LV> \
  -c /root/debian-<VERSION>-amd64-netinst.iso --vnc --noautoconsole --os-type linux \
  --os-variant debianLenny --accelerate --network=bridge:bridge0 --hvm -k de

To monitor the process use virt-viewer -c qemu+ssh://:/system .

RHCS setup

The next step is the setup of RHCS itself. Again, first things first, the software: apt-get install redhat-cluster-suite. This pulls quite a number of services which are not needed in our example:

invoke-rc.d nfs-kernel-server stop
invoke-rc.d nfs-common stop
invoke-rc.d portmap stop
update-rc.d -f nfs-kernel-server remove
update-rc.d -f nfs-common remove
update-rc.d -f portmap remove

Btw., system-config-cluster is not available for Lenny, but our Philipp Hübner has created a backport:

wget --no-check-certificate https://www.credativ.com/~phu/lenny-backports/system-config-cluster/system-config-cluster_1.0.53-1_all.deb
dpkg -i system-config-cluster_1.0.53-1_all.deb
apt-get -f install
apt-get install xauth

In order to have locking on the LVM cluster, you now need to modify /etc/lvm/lvm.conf: check for the global part.

 locking_type = 3

With the newer kernels the module lock_dlm also vanished, so CMAN init script must be modified: comment out the line modprobe lock_dlm 2>&1 || return 1. Additionally, RHCS 2 only supports XEN, so for libvirt you need to load the resource handler vm.sh.

wget --no-check-certificate https:///www.credativ.com/~phu/vm.sh -O /usr/share/cluster/vm.sh
chmod +x /usr/share/cluster/vm.sh

RHCS itself is called via

/etc/init.d/cman start
/etc/init.d/clvm start
/etc/init.d/rgmanager start

Fencing

Fencing describes the automagical neutralization of nodes which cease to function properly. In our example we use a power plug which can be controlled via network, NETIO-230A. Currently there is no real fence agent available for the device, but the python library Python-Bibliothek offers the necessary background to quickly write one.

Closing words

This howto has shown the setup of RHCS on Debian in easy steps - but of course, the correct steps depend very much on the targeted services, so this is just an example. If you need help just ask - Open Source HA solutions are our speciality, and we offer services and support for KVM virtualization as part of our day to day business.

In May, Consultants from credativ GmbH will be holding a 3 day advanced system and network administration workshop at the Open Source School in Munich.

Training specifics (subject to modifications!):

• Kerberos: This training covers the Kerberos authentification protocol, which can handle a range of services and operating systems transparently. The use of tickets makes single-sign-in possible; so a user can access all services with a unique log in. The training will be aimed at network and system administrators who wish to roll out Kerberos in their business or administrative network; it will also cover the installation and management of Kerberos, as well as the integration of services and client programs.
  

  
  When: 03-05/05/2010 and 13-15/09/2010

• Spam and Virus Defense: This training will clarify the integration and fine tuning of open source based services Postfix, Amavis and SpamAssassin, which protect a network from unnecessary strain due to spam mail or malware. This training will be geared at administrators who wish to secure their company's email systems against spam and viruses.
  

  
  When: 26-28/05/2010 and 18-20/10/2010

• Samba in heterogenous networks: This training concerns Samba as a replacement for Windows servers for smooth integration for both Windows clients in unix-based networks, and Linux servers in Windows-based networks. The training is directed at administrators wanting to migrate a Windows network completely or partly to Linux with the help of Samba. The goal of the training is the management and administration of LDAP-based primary/backup domain controller setups.
  

  
  When: 30/06-02/07/2010
  

The training will take place at the Open Source School in Munich city centre, Amalienstrasse 77. Applications can be made via the Open Source School website or by faxing this form. For further information contact Michael Banck.

Further dates for your diary: 21-23 April - PostgreSQL training will be carried out by credativ experts at the Linuxhotel Linuxhotel in Essen.