From Kerrighed

This page describes the installation steps for getting Kerrighed 2.2.x running on your cluster.

Contents

[edit] Presentation

Once installed, the Kerrighed OS is made of the following parts:

  • a patched Linux kernel,
  • a Linux module,
  • some user-level tools,
  • scripts to start/stop Kerrighed service,
  • a library, libkerrighed.

[edit] Getting Kerrighed

  • Patch, module, tools, services and library sources are available in a tarball on INRIA's GForge.
  • Linux 2.6.20 vanillas sources are also needed.

[edit] Pre-requisites

The following software is required Kerrighed to compile:

  • automake: >=1.9
  • autoconf: >=2.59
  • libtool
  • rsync
  • bzip2
  • gcc: 3.3.x is the recommended compiler for Linux Kernel.

Kerrighed is reported to build with 4.1 series.

Optional features requires the following:

  • lsb_release: to install Kerrighed startup scripts into /etc/init.d
  • xmlto: a tool to create manpages from docbook

[edit] Quick Compilation and Installation

[edit] Cluster settings

In order to use Kerrighed on your cluster, all nodes should share a common complete file system tree. To do so, you can use:

  • NFS Root
  • A shared external storage system (SAN, NAS)
  • Etc.

Currently, we strongly recommand to use NFSROOT. See the NFSROOT quick tutorial to set-up your cluster this way.

[edit] Installing Kerrighed

[edit] Installing Kerrighed from the tarball

  • We assume you are using NFSROOT and on your tftp server:
    • /NFSROOT/kerrighed is the chrootable filesystem,
    • /srv/tftp contains tftp files.
  • We assume that your working dir is /usr/src.
  • The following steps must be done as root.
  • On your tftp server, chroot to the directory hosting your cluster file system 
chroot /NFSROOT/kerrighed
  • Decompress the Tarballs 
cd /usr/src
tar zxf /path/to/kerrighed-2.2.x.tar.gz
tar jxf /path/to/linux-2.6.20.tar.bz2
  • Configure the Sources 
cd kerrighed-2.2.x
./configure --with-kernel=/usr/src/linux-2.6.20
  • Patch the kernel 
make patch
  • Configure the kernel

Patch comes with a default configuration which may not suit your needs but gives you a running Kerrighed configuration. There are different ways to configure the kernel options:

    • Run one of make *config in the kernel source tree.
    • Run make kernel-config KERNEL_CONFIG=/path/to/a/configfile.

For some hints on kernel options, have a look at Kernel Options section below.

  • Build the sources 
make kernel
make
  • Install all, as user root 
make kernel-install
make install
  • Check the installation. You should have now the following dir/files installed:
/boot/vmlinuz-2.6.20-krg Kerrighed kernel
/boot/System.map Kerrighed kernel symbol table
/lib/modules/2.6.20-krg Kerrighed modules
/etc/init.d/kerrighed Kerrighed service script *
/etc/default/kerrighed Service configuration
/usr/local/share/man Manpages
/usr/local/bin/krgadm Cluster administration tool
/usr/local/bin/krgcapset Process capabilities tool
/usr/local/bin/migrate Process migration tool
/usr/local/lib/libkerrighed-* Kerrighed library
/usr/local/include/kerrighed Kerrighed library headers

* This script prepare the node to integrate a cluster, but does not start the cluster. See krgadm (1)

[edit] Installing Kerrighed from the Debian package 

wget -O /etc/apt/sources.list.d/kerrighed.list http://kerrighed.gforge.inria.fr/debian/kerrighed.list
  • Update packages list: 
apt-get update
  • Install one of kerrighed or kerrighed-dev packages:
    • kerrighed installs kernel, module, library and tools,
    • kerrighed-dev installs kernel and module with debugging symbols, library plus headers and static library and tools 
apt-get install kerrighed
apt-get install kerrighed-dev


[edit] Kerrighed configuration

  • We assume you have a 4 nodes cluster with node names ranging from ssi1 to ssi4.
  • On your tftp server, copy the kernel and ramdisk (if any) in /srv/tftp: 
cp /NFSROOT/kerrighed/boot/vmlinuz-2.6.20-krg /srv/tftp
cp /NFSROOT/kerrighed/boot/initrd-2.6.20-krg /srv/tftp
  • Now, edit files in /srv/tftp/grub/. For example, /srv/tftp/grub/ssi1 may contain:
    • Note 1: default instruction tells which boot entry the node should boot by default, starting from 0.
    • Note 2: node_id bootparam must be unique for each node in your cluster. So, you must write a different Grub menu for each node. 
default 1
timeout 1

title Debian
root (nd)
kernel /vmlinuz-xxx root=/dev/nfs ip=dhcp nfsroot=192.168.0.1:/NFSROOT/kerrighed
initrd /initrd.img-xxx

title Debian
root (nd)
kernel /vmlinuz-2.6.20-krg root=/dev/nfs ip=dhcp nfsroot=192.168.0.1:/NFSROOT/kerrighed node_id=0 session_id=1
initrd /initrd.img-2.6.11-krg

[edit] Hostname hint

To setup hostname on your cluster nodes, you can use a DNS or setup /etc/hostname file. In the latter case, if you use a shared root filesystem, this file will be the same on each node, so you have to use a hook to add, for instance, last digit of the IP address to the hostname.

On a Debian system, you may edit /etc/init.d/hostname as follow (only do_start function is modified): 

do_start () {
       ADDR=`ifconfig eth0 | grep "inet addr" | cut -d ":" -f 2 | cut -d " " -f 1`
       HOSTNAME=`grep $ADDR /etc/hosts | tr -s ' ' | cut -f2 | sed 's/\([a-zA-Z_-]*\)\..*/\1/'`

       [ "$VERBOSE" != no ] && log_action_begin_msg "Setting hostname to '$HOSTNAME'"
       hostname "$HOSTNAME"
       ES=$?
       [ "$VERBOSE" != no ] && log_action_end_msg $ES
       exit $ES
}

[edit] Advanced configuration

[edit] Kerrighed configuration

The configure script provides a lot of options to configure the build and installation process.

[edit] Generic configure options

By default, make install will install the package's files in /usr/local/bin, /usr/local/man, etc. You can specify an installation prefix other than /usr/local by giving configure the option --prefix=PATH.

configure recognizes the following options to control how it operates. 

--cache-file=FILE
    Use and save the results of the tests in FILE instead of
    ./config.cache.  Set FILE to /dev/null to disable caching, for
    debugging configure.

--help
    Print a summary of the options to configure, and exit.

--quiet
--silent
-q
    Do not print messages saying which checks are being made.

--srcdir=DIR
    Look for the package's source code in directory DIR.  Usually
    configure can determine that directory automatically.

--version
    Print the version of Autoconf used to generate the configure
    script, and exit.

[edit] Kerrighed specific options

Following options can be passed to the toplevel configure script: 

--disable-linux-check
    Check for the Linux sources to be compatible with
    Kerrighed (!!! disable at your own risk !!!) [default=enable]

--disable-module
    Patch the kernel with Kerrighed and build the module [default=enable]

--disable-libkerrighed
    Disable libkerrighed [default=enable]

--disable-tools
    Disable tools [default=enable]

--disable-service
    Do not install kerrighed service [default: enable if lsb found]

--with-kernel=PATH
    Path to kernel source [default=/lib/modules/`uname -r`/source]

--with-kernel-build=PATH
    Path to kernel build [default=/lib/modules/`uname -r`/build]

--disable-kernel-defconfig
    Do not use default config for kernel compilation [default=enable]

--with-kernel-config=PATH
    Path to a kernel .config file. Sample files are provided in 'samples'
    directory. Implies --disable-kernel-defconfig [default=none]

[edit] Kernel options

If you want to edit kernel options, note that the following are currently broken with Kerrighed:

  • General setup -> IPC Namespaces
  • Processor type and features -> Preemption Model -> Voluntary Kernel Preemption
  • Processor type and features -> Preemption Model -> Preemptible Kernel
  • Networking -> IrDA (infrared) subsystem support
  • Security options -> Enable access key retention support
  • Networking -> Networking options -> The Ipv6 protocol

Note: don't forget to add NIC driver and, if you plan to use NFSROOT, include it in the kernel, not as module.

[edit] Using Kerrighed

Now that Kerrighed is installed and configured on your system, have a look at the user manual.

Retrieved from "http://www.kerrighed.org/wiki/index.php/Installing_Kerrighed_2.2.0"