From Kerrighed

This page describes the installation steps for getting Kerrighed 2.0.0 running on your cluster.

Contents

Presentation

Once installed, the Kerrighed OS is made of the following parts:

  • a patched Linux kernel,
  • a Linux module,
  • some user-level tools,
  • scripts to start/stop Kerrighed service,
  • a library, libkerrighed.

Getting Kerrighed

  • Patch, module, tools, services and library sources are available in a tarball.
  • Linux 2.6.11 vanillas sources are also needed.

Pre-requisites

The following software is required Kerrighed to compile:

  • automake: >=1.9
  • autoconf: >=2.59
  • libtool
  • gcc: 3.3.x is the recommended compiler for Linux Kernel.

Kerrighed is reported to build with 4.1 series.

Optional features requires the following:

  • lsb_release: to install Kerrighed startup scripts into /etc/init.d
  • xmlto: a tool to create manpages from docbook

Quick Compilation and Installation

Cluster settings

In order to use Kerrighed on your cluster, all nodes should share a common complete file system tree. To do so, you can use:

  • NFS Root
  • A shared external storage system (SAN, NAS)
  • Etc.

Currently, we strongly recommand to use NFSROOT. See the NFSROOT quick tutorial to set-up your cluster this way.

Installing Kerrighed

Installing Kerrighed from the tarball

  • We assume you are using NFSROOT and on your tftp server:
    • /NFSROOT/kerrighed is the chrootable filesystem,
    • /srv/tftp contains tftp files.
  • We assume that your working dir is /usr/src.
  • The following steps must be done as root.
  • On your tftp server, chroot to the directory hosting your cluster file system 
chroot /NFSROOT/kerrighed
  • Decompress the Tarballs 
cd /usr/src
wget http://gforge.inria.fr/frs/download.php/1916/kerrighed-2.0.0.tar.gz
wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.11.tar.bz2
tar zxf kerrighed-2.0.0.tar.gz
tar jxf linux-2.6.11.tar.bz2
  • Configure the Sources 
cd kerrighed-2.0.0
./configure --with-kernel=/usr/src/linux-2.6.11
  • Build the sources 
make kernel
make
  • Install all, as user root 
make kernel-install
make install
  • Check the installation. You should have now the following dir/files installed:
/boot/vmlinuz-2.6.11-krg Kerrighed kernel
/boot/System.map Kerrighed kernel symbol table
/lib/modules/2.6.11-krg Kerrighed modules
/etc/init.d/kerrighed Kerrighed service script *
/etc/default/kerrighed Service configuration
/usr/local/share/man Manpages
/usr/local/bin/krgadm Cluster administration tool
/usr/local/bin/krgcapset Process capabilities tool
/usr/local/bin/migrate Process migration tool
/usr/local/lib/libkerrighed-* Kerrighed library
/usr/local/include/kerrighed Kerrighed library headers

* This script prepare the node to integrate a cluster, but does not start the cluster. See krgadm (1)

Installing Kerrighed from the Debian package 

wget -O /etc/apt/sources.list.d/kerrighed.list http://kerrighed.gforge.inria.fr/debian/kerrighed.list
  • Update packages list: 
apt-get update
  • Install one of kerrighed or kerrighed-dev packages:
    • kerrighed installs kernel, module, library and tools,
    • kerrighed-dev installs kernel and module with debugging symbols, library plus headers and static library and tools 
apt-get install kerrighed
apt-get install kerrighed-dev


Kerrighed configuration

  • We assume you have a 4 nodes cluster with node names ranging from ssi1 to ssi4.
  • On your tftp serveur, copy the kernel and ramdisk (if any) in /srv/tftp: 
cp /NFSROOT/kerrighed/boot/vmlinuz-2.6.11-krg /srv/tftp
cp /NFSROOT/kerrighed/boot/initrd-2.6.11-krg /srv/tftp
  • Now, edit files in /srv/tftp/grub/. For example, /srv/tftp/grub/ssi1 may contain:
    • Note 1: default instruction tells which boot entry the node should boot by default, starting from 0.
    • Note 2: node_id bootparam must be unique for each node in your cluster. So, you must write a different Grub menu for each node. 
default 1
timeout 1

title Debian
root (nd)
kernel /vmlinuz-xxx root=/dev/nfs ip=dhcp nfsroot=192.168.0.1:/NFSROOT/kerrighed
initrd /initrd.img-xxx

title Debian
root (nd)
kernel /vmlinuz-2.6.11-krg root=/dev/nfs ip=dhcp nfsroot=192.168.0.1:/NFSROOT/kerrighed node_id=0
initrd /initrd.img-2.6.11-krg
  • Edit /NFSROOT/kerrighed/etc/kerrighed_nodes:
    • session is an unique cluster identifier. 1 is a good value in most cases.
    • nbmin is the number of nodes in your cluster
    • For each node, add a line with <node name>:<node id>:<network interface to use>
    • More informations in the kerrighed_nodes man page 
session=1
nbmin=4
ssi1:0:eth0
ssi2:1:eth0
ssi3:2:eth0
ssi4:3:eth0


Hostname hint

To setup hostname on your cluster nodes, you can use a DNS a setup /etc/hostname file. In the latter case, if you use a shared root filesystem, this file will be the same on each node, so you have to use a hook to add, for instance, last digit of the IP address to the hostname.

On a Debian system, you may edit /etc/init.d/hostname as follow (only do_start function is modified): 

do_start () {
       ADDR=`ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2 | cut -d " " -f 1`
       HOSTNAME=`grep $ADDR /etc/hosts | tr -s ' ' | cut -f2 | sed 's/\([a-zA-Z_-]*\)\..*/\1/'`

       [ "$VERBOSE" != no ] && log_action_begin_msg "Setting hostname to '$HOSTNAME'"
       hostname "$HOSTNAME"
       ES=$?
       [ "$VERBOSE" != no ] && log_action_end_msg $ES
       exit $ES
}

Advanced configuration

Kerrighed configuration

The configure script provides a lot of options to configure the build and installation process.

Generic configure options

By default, make install will install the package's files in /usr/local/bin, /usr/local/man, etc. You can specify an installation prefix other than /usr/local by giving configure the option --prefix=PATH.

configure recognizes the following options to control how it operates. 

--cache-file=FILE
    Use and save the results of the tests in FILE instead of
    ./config.cache.  Set FILE to /dev/null to disable caching, for
    debugging configure.

--help
    Print a summary of the options to configure, and exit.

--quiet
--silent
-q
    Do not print messages saying which checks are being made.

--srcdir=DIR
    Look for the package's source code in directory DIR.  Usually
    configure can determine that directory automatically.

--version
    Print the version of Autoconf used to generate the configure
    script, and exit.

Kerrighed specific options

Following options can be passed to the toplevel configure script: 

--disable-linux-check
    Check for the Linux sources to be compatible with
    Kerrighed (!!! disable at your own risk !!!) [default=enable]

--disable-module
    Patch the kernel with Kerrighed and build the module [default=enable]

--disable-libkerrighed
    Disable libkerrighed [default=enable]

--disable-tools
    Disable tools [default=enable]

--disable-service
    Do not install kerrighed service [default: enable if lsb found]

--with-kernel=PATH
    Path to kernel source [default=/lib/modules/`uname -r`/source]

--with-kernel-build=PATH
    Path to kernel build [default=/lib/modules/`uname -r`/build]

--disable-kernel-defconfig
    Do not use default config for kernel compilation [default=enable]

--with-kernel-config=PATH
    Path to a kernel .config file. Sample files are provided in 'samples'
    directory. Implies --disable-kernel-defconfig [default=none]

Kernel configuration

For kernel configuration, you can:

  • use the default config provided with Kerrighed (default)
  • use an existing .config file using configure option --with-kernel-config.
  • use one of Linux configuration tools (make menuconfig, make xconfig, etc.).

Note: don't forget to add NIC driver and, if you plan to use NFSROOT, include it in the kernel, not as module.

Using Kerrighed

Now that Kerrighed is installed and configured on your system, have a look at the user manual.

Retrieved from "http://www.kerrighed.org/wiki/index.php/Installing_Kerrighed_2.0.0"