February 2010 Archives

credativ employee, Alexander Wirt, is due to give a presentation at the German Chemnitz Linux Days about Single Sign On with Kerberos. Besides an introduction into configuration of Kerberos, the talk will also focus on the configuration of its various services.

Kerberos is an authentication protocol which enables an admin to incorporate services and an operating system transparently into an existing setup. This makes Single Sign On possible: the user only has to enter his/her credentials once and thereafter can access any secured services and websites which support Kerberos without having to enter them again.

The Kerberos Single Sign On approach will be described by Alexander Wirt, credativ's expert on this topic, during a talk at the Chemnitz Linux Days due to take place in March. Besides the basic introduction to Kerberos based on Heimdal, he will also explain how to configure services such as SSH, Apache and IMAP. The topic of this talk will be very close to real-world usage, thus it should enable members of the audience to try it out easily themselves on their own networks.The talk will take place in German on March 13th 2010 at 15:00 in Room V4.

The PostgreSQL project just released the Alpha 4 of its upcoming PostgreSQL 9.0.

The Alpha4 version of the upcoming PostgreSQL 9.0 release is ready for download. It is planned that Alpha4 will be the last Alpha version before the Beta release cycle for PostgreSQL 9.0. Some highlights of this release are:

• Reworked LISTEN/NOTIFY infrastructure: the performance has improved massively compared to the old table-based implementation, due to a pure main memory solution. Additionally, the new solution supports so called "Payloads" which makes it possible to transport messages.


• Streaming Replication: an integrated solution for replication which has noticeable lower latency times than the usual, WAL-shipping-based solutions.


• Procedural code with plpqsql and plperl can now be executed with the DO statement without the need to call a CREATE FUNCTION first.

You are very much welcome to download and test the Alpha version and play with it. The developers are interested in Bugs and test results; you can find the work flow to publish these outlined in their Wiki.

The series "PostgreSQL Optimiser Bits" will introduce the strategies and highlights of the PostgreSQL optimiser. We start today with a new feature of PostgreSQL 8.4: Semi and Anti Joins.

Since version 8.4, PostgreSQL has been offering a new optimisation strategy for the optimisation of certain queries: Semi and Anti Joins.

A Semi Join is a specific form of a join, which only takes the keys of relation a into account if these are also present in the associated table b. An Anti Join is the negative form of a Semi Join: that is, a key picked in table a will be taken into account if it is not present in table b.

To summarize, Semi and Anti Joins are specific forms of a join which only take certain keys on the left side into account - where queries want to make sure certain keys exist, but are not concerned with the content of the key itself. This behaviour is already widely known in Object Relation Mappers (ORM) which formulate such queries using EXIST() or NOT EXIST().

Compared to PostgreSQL 8.3 the same query is possible with a much simpler and more efficient query plan. The following simple example shows this improvement: take two tables, a, b and an EXIST() query. A certain set of data from a is to be found which has its equivalent a.id2 = b.id in b. Of course, this aim can also be accomplished by one single join, however, this example shows the improvements of the optimizer solving this query.

EXPLAIN SELECT id FROM a WHERE a.id = 200 AND EXISTS(SELECT id FROM b WHERE a.id2 = b.id);

The optimiser in PostgreSQL in 8.3 determines the following plan for this example. Keep in mind that both tables a and b each have an index on the column id and id2.

                                QUERY PLAN
--------------------------------------------------------------------------
 Index Scan using a_id_idx on a  (cost=0.00..8355.27 rows=503 width=4)
   Index Cond: (id = 200)
   Filter: (subplan)
   SubPlan
     ->  Index Scan using b_id_idx on b  (cost=0.00..8.27 rows=1 width=4)
           Index Cond: ($0 = id)

In contrast, in PostgreSQL 8.4 the optimizer can use a hash Semi Join:

                                QUERY PLAN
---------------------------------------------------------------------------
 Hash Semi Join  (cost=27.52..78.16 rows=969 width=4)
   Hash Cond: (a.id2 = b.id)
   ->  Index Scan using a_id_idx on a  (cost=0.00..37.32 rows=969 width=8)
         Index Cond: (id = 200)
   ->  Hash  (cost=15.01..15.01 rows=1001 width=4)
         ->  Seq Scan on b  (cost=0.00..15.01 rows=1001 width=4)

The reduced costs of this query plan are more than obvious - and lower costs mean fewer I/O accesses. So, in future a more detailed analysis of such queries is worth a look.

The Debian project has announced that its internal DNS infrastructure is gradually moving over to DNSSEC. Thus from now on, all DNS answers for debian.com, amongst others, will be digitally signed to verify their authenticity.

The Domain Name System (DNS) is one of the core components of the Internet. However, the initial design of DNS is vulnerable against some quite serious attacks, among them cache poisoning which means faking of DNS answers. To avoid this problem, DNSSEC was introduced (DNSSEC in Wikipedia, see also dnssec.net). DNSSEC is an enhancement of the default DNS protocol which makes it possible to sign and thus verify DNS answers. The introduction of DNSSEC into the existing worldwide Internet infrastructure is proving to be slow, due to the complexity and amount of work involved; there are so far only a few top level domains (TLDs) and domains of large projects and companies providing signed answers.

The Debian project has now decided to introduce DNSSEC step by step, so that all project domains will provide verified DNS answers. Firstly, all debian.net and debian.com domains will be signed, and thereafter the collected experience will be used to sign the other domains and sub-domains.

One of the problems the Debian project is facing in the use of DNSSEC is that as yet they do not have the signatures by the TLDs that Debian uses, without which there is no third party to verify the Debian keys. To get around this, Debian will publish the DNSSEC keys via the DNSSEC Look-aside Validation Registry of ISC. This will mean Debian keys can be verified even for TLDs which have not yet introduced a DNSSEC infrastructure.

Moving the Debian project to DNSSEC improves the security of the Debian part of the internet. Additionally, the experience acquired in the changing over of such a large, worldwide and multi-domain project should help other projects of a similar size - and hopefully encourage them to follow suit.

One of the fastest ways to restrict the access to a web server path is to use .htaccess files. These are placed in the to-be-restricted path and contain the information how to restrict the access. A simple example is given below:

AuthType Digest
AuthName "SecretDir"
AuthUserFile /etc/apache2/passwords/secretdir.htpasswd
Require valid-user

The .htpasswd file contains the user login data for accessing the path and thus must not be placed in a path which is exported by the web server! It is created using its own command line tool: htdigest -c /etc/apache2/passwords/secretdir.htpasswd SecretDir $USERNAME Note, however, that the Directory part of the web site configuration (for example in ) must contain the option AllowOverride AuthConfig. Also, since this examples uses the AuthType Digest, make sure the module mod_auth_digest is loaded.

Further information and a more detailed configuration explanation is given in the article Authentication, Authorization and Access Control in Apache's documentation.

A couple of hours ago Nokia and Intel announced that they plan to merge their mobile Linux platforms to create a common base: MeeGo.

Giants Intel and Nokia have had their own Linux platforms for some time; Nokia was developing Maemo while Intel pushed Moblin forward. These two were battling in an already highly competitive market together with Android and LiMo - not to mention the proprietary solutions.Now Intel and Nokia have announced the merge of their two platforms into a new, combined platform: MeeGo.

"Moblin and maemo are merging! We are taking the best pieces from these two open source projects and are creating the MeeGo software platform."

In an economical sense, this is quite a sensible step. Intel and Nokia are two players who can defend themselves against even the likes of Google, Apple or Microsoft. However, in such a hard-fought market they do need partners or a new strategy at least, since both have so far failed to launch a solution with a larger market reach than their competitors. (There are few devices available with Maemo or Moblin.) They have the added advantage that both platforms were technically similar, so the merge enables them to save resources. But the question remains as to whether Intel and Nokia will come into competition with each other on the device market, as MeeGo will be run on a variety of devices, from mobile phones to TVs.

From a technical perspective, it is interesting to see that MeeGo will use Qt as the main graphics library. This framework, currently owned by Nokia, is used to program GUIs; among others, it forms the base for KDE. credativ employees often work with Qt and, being familiar with the advantages of the framework also offer Qt-support to customers.  With Qt, MeeGo is based upon a well-proven and modern, graphical framework. Besides, it has been rumoured that MeeGo will be based upon RPM. There are references to "kickstart" files in the developer section, which suggest MeeGo will be close to distributions such as Fedora, OpenSuse or RHEL.

Besides the technical and economic perspective, the community question is significant for any new platform. While it is said that LiMu leaves the community out in total, and Android is criticised for not merging their Kernel patches upstream, in contrast, it is planned that MeeGo will be developed under the umbrella of the Linux Foundation, which is well known to the Community and also employs some  outstanding developers.

Time will tell if MeeGo will stand its ground - as yet, there are no devices available and the competition is strong... but it will definitely be interesting to watch!

One of the many advantages of vim is the possibility to use syntax highlighting: different terms and chars are displayed in different colors according to their meaning and function.The more colours are available, the better - however, in many cases vim falls back to 8 or 16 colors by default. This can be quickly changed with an entry in ~/.vimrc:

if &term!="xterm"
   set t_Co=256            " use 265 colors in vim
   colorscheme desert256   " an appropriate color scheme
endif

With this configuration, vim from now on uses 256 colors - and the second line makes sure you a color scheme which actually uses that many colors: desert256 - but make sure you copy the color scheme file to ~/.vim/colors/ before! The if-query ensures that the configuration is not loaded on ttys where you usually don't have 256 colors.

By the way: an overview of all kinds of color schemes with "screenshots" for different programming languages can be found at the vimcolorschemetest project web page.

The Skolelinux team has released version 5.0 of its popular school server, which is now based on Debian Lenny.

The distribution Skolelinux, also known as Debian-Edu, is a Debian version adapted for running school networks with Linux.  It enables even non-geek users to quickly set up a central school server with a terminal server, thin clients, workstations and laptops as desktop computers.

With the release of version 5.0, Skolelinux was lifted to the current Debian software base. Besides providing enhanced hardware support, it now updates the software to recent versions, making the new release more appealing to pupils and schools. Some other technical highlights of this version include:

• GNOME is now supported alongside KDE;


• The pupils' destkop has been enhanced with links to school software like GCompris, Kalzium, KGeography, KMplot, KStars, Stopmotion and the OpenOffice Suite;


• Improved LTSP server configuration:
  • Besides the traditional thin clients, diskless workstations are now supported out of the box; manual tweaking is no longer necessary;
  
  
  • The new PXE start menu allows diskless workstations to boot via network or via local media;
  
  
  • Software running locally on diskless workstations comes straight from the server, where it can be more easily maintained.


• Improved documentation, now translated into German, Italian and Norwegian;


• Improved and simplified user- and machine-administration tool LWAT (LDAP Web-based Administration Tool);


• Improved browser support with free software products like Gnash, Java and other plugins;


• Improved monitoring, now reporting on the status of all machines which are automatically connected to the network;


• Improved audio and multimedia experience in PulseAudio, next to Alsa and OSS.

Philipp Hübner, voluntary worker of the Skolelinux project and employee here at credativ underlines the modern approach of the new version:

"Skolelinux made a huge leap forward with the new release: the seamless, out of the box integration of the Diskless Workstation makes Skolelinux perfect for the challenges of today, as it evolves to keep up with the constantly growing performance of computers. This way, the performance of modern workstations is perfectly combined with the low maintainance work of thin clients."

Skolelinux is now used by milions of pupils across various countries, including Spain, Norway and Germany. In 2009, credativ Germany supervised the successful completion of a Skolelinux evaluation project, a solution which is increasingly adopted in schools today. As a certified support partner, credativ is able to offer professional support services in the education sector.We congratulate Skolelinux for this release: well done, we wish you all the best for the future!

This week the KDE project released version 4.4 of its software compilation. Besides new scientific programs, this version focuses on stability and enhancements of the current programs and functions.

The KDE project released version 4.4 of its "KDE Software Compilation" (KDE SC) package. The new version is especially striking for its attention to detail in the user experience:

An overview of all new features can be found in the Feature Guide, which has more images and videos. Here are some of the most significant enhancements:

• Substantial improvements of the semantic search system Nepomuk: the new default storage backend is a lot faster than the old one.


• Plasma has advanced: Widgets can now be shared with other users over the network and the management and handling of external storage devices has been reworked.


• New programs: besides the blog application Blogilo, KDE SC 4.4 also comes with the scientific programs Cantor and Rocs.


• The development platform for KDE Software now has the new authentication framework KAuth, which allows more specific and secure allocation of user rights.

Another new feature which I personally like, is basic support for automatic window sizing - almost basic tiling support.  For example, whenever you drag a window to the left window, it is automatically resized to cover the left half of the screen. This is particularly useful when working with several windows at a time on a huge screen.

Compared with previous KDE versions, there are less new features in this release. The developers have really focused on fixing bugs and improving stability. This is one of the reasons why users who are still running KDE 3.x are advised to change over to KDE 4.x now; the distributions will ship KDE 4.4 soon anyway.

During the release of KDE SC 4.4, the main website of the KDE project was also re-designed. kde.org was completely reworked and now includes information from external sources as opendesktop.org. This means interested users have access to much better and more detailed information about the available software.Congrats on the new release from credativ!

PostgreSQL is taking some big steps forward this year. The publishing of version 9.0 is just around the corner, while some of the older versions are coming to the end of their lifetime.

PostgreSQL 9.0

2010 will see PostgreSQL release its first major new version for a long time: version 9.0. The release of version 9.0 is an important milestone in the evolution of PostgreSQL. Integral to this release are new features such as the operation of standby servers in read-only mode (hot standby) and an integrated replication solution.

Hot Standby

Hot standby will allow a PostgreSQL instance to receive read requests on so-called standby nodes. The basic principle is the same as that included since version 8.0 under the name PITR (Point In Time Recovery) or WAL-Shipping. At regular intervals a copy of the database complete with transaction logs is generated (known as the Write Ahead Log or WAL), so that the standby nodes can be kept up to date with changes in the master database. In practice, this means incrementally applying all changes that were made on the master database from the point when the standby node was created. This was implemented as warm standby in previous versions, i.e. the database contained within a standby node could not be used by applications. However, with hot standby, it is possible to execute transactions on the node as long as they do not contain write operations. This is especially useful for high availability systems or analyses that can be run on separate nodes.

Streaming Replication - inbuilt asynchronous replication

For a long time in the PostgreSQL community, it was widely thought amongst developers that the infrastructure of an integrated replication system was difficult to maintain due to the complex requirements and variety of deployment scenarios. Therefore the flexibility and security expected of such solutions has been implemented in various specialised external projects. In recent years however, extensive communication with users has led to a large proportion of the desired functionality being implemented within PostgreSQL, mostly in the area of high availability. Thanks to this, an integrated solution is no longer just a dream, even for systems containing hundreds of gigabytes of data. Furthermore, the availability of an integrated replication solution is a critical factor for many data centres when choosing a database management system. Streaming replication means that PostgreSQL can now offer an integrated solution for asynchronous replication of a primary database server (read- and writeable) to multiple additional secondary servers (read only). This functionality, based in part on the infrastructure implemented for WAL-Shipping, has made possible the replication of transactions in much smaller intervals. (Data is sent directly from the primary to the secondary server, hence the name "streaming"). Moreover, streaming replication permits the simple implementation of PostgreSQL replication clusters with multiple nodes. Whilst this is already possible with the existing hot-standby solution, it is much more complicated. Since the replicating data is based upon information from the WAL, this solution is extremely robust. Deployment scenarios such as partially replicated databases or modified database schemas are not currently possible on each replicated node, although these requirements are still achievable through the use of solutions such as Slony-I, Londiste or Bucardo.

Farewell to PostgreSQL 7.4, 8.0 and 8.1

2010 will herald the end of support for some versions of PostgreSQL. For the first time, three main versions are due to be phased out in the same year:

• PostgreSQL 7.4, Juli 2010
• PostgreSQL 8.0, Juli 2010
• PostgreSQL 8.1, November 2010

Support for PostgreSQL 8.0 and 8.1 on Windows was discontinued with the release of PostgreSQL 8.3 in February 2008. PostgreSQL 8.0 was the first release that could run natively on Windows, with many bugs being patched during development that could no longer be backported to older versions. So for quite some time now, Windows users have had to use at least PostgreSQL 8.2. We are now officially coming to the end of support for all other platforms, and also the last of the 7 series releases; PostgreSQL 7.4 is finally being phased out after 7 years. "Phased out" in PostgreSQL terms means that, primarily, no further binary packages or releases will be made and no further complex fixes will be ported, although the source code will continue to be available. As a rule, the PostgreSQL development team limit the lifetime of a main release to five years. However, the Windows variants of PostgreSQL 8.0 and 8.1 are proof that the lifetime of releases for single platforms can be shortened. The Release Policy can be found in the developer wiki on the PostgreSQL project site.

Outlook

Although PostgreSQL 9.0 is not yet finished, hot standby can be tested with version 8.5alpha3. Incidentally, the current alpha version is still named after the developer's branch 8.5, as it was named before the decision was made to move to version 9.0. Version 9.0alpha4 can be expected by late February, and should also include streaming replication. For those interested in testing, we are planning a guide with the title "How To Beta Test", which provides some guidelines for testing and feedback.

The 2010 Debian Bug Squashing Party turned out to be a great success: around 200 bugs were fixed, ready for the forthcoming version of Debian.

The weekend of 22-24 January saw this year's Bug Squashing Party hosted yet again by credativ.  The aim of the weekend was to find and fix bugs in the next Debian release.  The results were as follows:

Work on Debian	Results
Installed Patches	5
Fixed Bugs	44
Non-critical Bugs	28
Completely removed Packages	87
Packages removed from Testing	29

Altogether that gives a grand total of 200 bugs.  In addition, information was gathered on a further 100 bugs, which will help when they come to be fixed.  A lot of time was also spent on quality assurance, an under appreciated but very important job.More gossip: it is rumoured that backports.org is not far off becoming an official Debian project... and in the meantime it will be launching a brand new website running on ikiwiki.

Our "guests" from far and wide were more than happy with the BSP party: Steve McIntyre: (Debian project leader)

Thanks to the folks at credativ for hosting and participating in the BSP - we got a huge amount of work done towards the next release and had a great time doing it!

Stefano Zacchiroli:

My 1st Mönchengladbach BSP, won't be the last! Lots of cool people and hacking, and I've enjoyed my 1st "traditional" Formorer's chilli too :).

credativ would like to thank all those who came and got involved - now you can lean back, relax and enjoy the photos of the event.

The current RHEL/CentOS 5 package has one flaw: it was compiled without Sieve support. However, with a bit of rpm magic, the package can be rebuilt and produces an additional sieve package.
The current RHEL/CentOS 5 version has a rather old dovecot, 1.0.7. Even worse, the plugin for Sieve wasn't included in this build. Of course, given the old version of dovecot, an update to a newer version with Sieve is worth a thought; however, there are situations where that is simply not an option.

In such cases you can still rebuild the old package with a modified rpm file: download the source RPM, install it with
rpm -Uvh dovecot-1.0.7-7.el5.src.rpm
get the diff from below and apply it to the spec file:
patch < dovecot.diff
Download the sources as given in the now modified spec file to your SOURCES directory, and rebuild the package:
rpmbuild -ba dovecot.spec
and welcome the new sieve plugin dovecot-sieve-1.0.4-7.x86_64.rpm. Install it and continue as usual. And as a small help for writing Sieve scripts: you can verify them on various online services like the one from the PHP Sieve library.

Be careful, however: you have to maintain this package on your own - especially when a dovecot update comes along or when the sieve plugin code is updated. Do bear in mind, though, that this information, as with all howtos, should be followed at your own discretion; it comes with no warranty, and might eat your cats.

And here is the patch for the spec file:

--- dovecot.old.spec    2010-03-11 09:59:38.598277799 +0100
+++ dovecot.spec        2010-03-11 09:58:08.639526842 +0100
@@ -1,7 +1,10 @@
 %define upstream 1.0.7
+%define sieve_upstream 1.0.4
 %define pkg_version 1.0.7
 %define my_release 7
 %define pkg_release %{my_release}%{?dist}
+%define pkg_sieve_version 1.0.4
+%define pkg_sieve_release %{my_release}%{?dist}
 
 Summary: Dovecot Secure imap server
 Name: dovecot

@@ -12,6 +15,7 @@
 
 %define build_postgres 1
 %define build_mysql 1
+%define sieve_name dovecot-sieve
 
 Source: http://dovecot.org/releases/%{name}-%{upstream}.tar.gz
 Source1: dovecot.init

@@ -22,6 +26,7 @@
 Source6: perfect_maildir.pl
 Source7: dovecot-REDHAT-FAQ.txt
 Source8: dovecot.sysconfig
+Source9: http://dovecot.org/releases/sieve/%{sieve_name}-%{sieve_upstream}.tar.gz
 Patch100: dovecot-1.0.7-default-settings.patch
 Patch102: dovecot-1.0.rc2-pam-setcred.patch
 Patch103: dovecot-1.0.beta2-mkcert-permissions.patch

@@ -80,6 +85,16 @@
 primarily in mind.  It also contains a small POP3 server.  It supports mail 
 in either of maildir or mbox formats.
 
+%package sieve
+Requires: %{name}
+Summary: CMU Sieve plugin for dovecot LDA
+Group: System Environment/Daemons
+Version: %{pkg_sieve_version}
+Release: %{pkg_sieve_release}
+
+%description sieve
+This package provides the CMU Sieve plugin for dovecot LDA.
+
 %prep
 %setup -q -n %{name}-%{upstream}

@@ -94,6 +109,8 @@
 %patch503 -p1 -b .CVE-2008-4577
 %patch504 -p1 -b .CVE-2008-4870
                               
+%setup -q -n %{name}-%{upstream} -D -T -a 9
+
 %build
 rm -f ./configure
 libtoolize -f

@@ -115,6 +132,16 @@
 
 make %{?_smp_mflags}
 
+cd %{sieve_name}-%{sieve_upstream}
+rm -f ./configure
+libtoolize -f
+autoreconf
+%configure                           \
+    INSTALL_DATA="install -c -p -m644" \
+    --with-dovecot=../
+
+make %{?_smp_mflags}
+
 %install
 rm -rf $RPM_BUILD_ROOT
 make install DESTDIR=$RPM_BUILD_ROOT

@@ -169,6 +196,11 @@
 mv $RPM_BUILD_ROOT%{docdir} $RPM_BUILD_ROOT%{docdir}-%{version}
 mkdir -p $RPM_BUILD_ROOT/var/lib/dovecot
 
+# dovecot-sieve
+pushd %{sieve_name}-%{sieve_upstream}
+make install DESTDIR=$RPM_BUILD_ROOT
+popd
+
 %pre
 /usr/sbin/useradd -c "dovecot" -u %{dovecot_uid} -s /sbin/nologin -r -d /usr/libexec/dovecot dovecot 2>/dev/null || :
 
@@ -243,6 +275,9 @@
 %attr(0750,root,dovecot) %{docdir}-%{version}/examples/mkcert.sh
 %attr(0750,dovecot,dovecot) %dir /var/lib/dovecot
 
+%files sieve
+%defattr(-,root,root)
+%{_libdir}/%{name}/lda/lib90_cmusieve_plugin.so
 
 %changelog
 * Mon Nov 24 2008 Michal Hlavinka <mhlavink@redhat.com> - 1.0.7-7