RSS icon News

From Kerrighed


Frequently Asked Questions

Which kernel options should I enable/disable ?

Once your kernel is patched, you can run make defconfig in kernel sources dir to generate a default configuration which is known to work. Kerrighed relative options are in Cluster support section. Disabled one, by default, are usually experimental.

Unfortunately, we can not test Kerrighed with all combination of Linux configurations. Nevertheless, the following options are known to be *broken* with Kerrighed:

  • (Until Kerrighed 2.4.x) General setup -> IPC Namespaces (CONFIG_IPC_NS)
  • Processor type and features -> Preemption Model -> Voluntary Kernel Preemption (CONFIG_PREEMPT_VOLUNTARY)
  • Processor type and features -> Preemption Model -> Preemptible Kernel (CONFIG_PREEMPT)
  • (Until Kerrighed 2.4.x) Processor type and features -> Non Uniform Memory Access (NUMA) Support (CONFIG_NUMA)
  • Executable file formats / Emulations -> IA32 Emulation (CONFIG_IA32_EMULATION)
  • Security options -> Enable access key retention support (CONFIG_KEYS)

Can I use Kerrighed with any version of Linux ?

No. Kerrighed is made of a patch to the Linux kernel plus some other tools. Hence it is designed on a particular version of a Linux kernel. You can find the Linux version to use the installation manual for each Kerrighed version.

Does Kerrighed support thread migration?

No. This feature was disabled early in the port from Linux 2.4.x to Linux 2.6.11. It should come back in the future. Please read this post for more details.

How do I know if the cluster is running? (aka How can I check that krgadm cluster start succeeded?)

Several ways:

  • On the cluster nodes, the kernel log contains a line like
    Kerrighed is running on XX nodes
    where XX is the number of nodes.
  • /proc/cpuinfo shows all CPUs of the cluster
  • /proc/meminfo shows all memory of the cluster
  • /proc/stat shows stats about all CPUs of the cluster

krgadm fails with segmentation fault

This is a known bug of krgadm in Kerrighed 2.3.0 and earlier. This is fixed in trunk and later versions.

krgadm cluster status tells me that no cluster is running, but krgadm cluster start seemed to work

krgadm cluster status is buggy in Kerrighed 2.3.0 and earlier. This is fixed in trunk and later versions.

My processes do not migrate!

Four main issues:

  • Check that they have the CAN_MIGRATE kerrighed capability. See also kerrighed_capabilities (7) and krgcapset (1).
  • Did you setup the scheduler? It is automatically setup by Kerrighed's init boot scripts if your distribution is supported. Otherwise ensure to run krg_legacy_scheduler as root on one node, after having started the cluster. See also SchedConfig.
  • (Since Kerrighed 3.0.0) Memory-mapped files are not on a shared filesystem. Typically, /bin, /lib, and /usr should be on a shared filesystem recognized as such by Kerrighed (ie NFS or OCFS2 for Kerrighed 3.0.0). Please consider using NFSROOT.
  • You did not read the documentation and you are not sharing files used by your processes between the nodes (For instance, /tmp, /var, /home, ...). Please consider using NFSROOT.

Otherwise some processes won't migrate because they use a currently unsupported feature (eg POSIX timers), or some of their cluster-wide resources (eg NFS files) could not be found on the target node.

My process does not fork remotely

My process cannot use remote memory

  • Check that the process was started with the USE_REMOTE_MEMORY Kerrighed capability. See also kerrighed_capabilities (7) and krgcapset (1).
  • (Kerrighed 2.4.x only) Check that memory overcommit is enabled (on all nodes). You can enable it on a node with:
    # echo 1 > /proc/sys/vm/overcommit_memory

Where are my keys ?

I don't know.