From Kerrighed

[edit] Latest news Nov 2008, kDFS at SC 2008, visit us at booth 2203 August 2008, kDFS is available as a standalone kernel module May 1 2008, kDFS design will be presented at Europar 2008 Apr 29 2008, Porting kDFS from 2.6.20 to 2.6.25 Apr 28 2008, kDFS and Posix FS suite :)

[edit] Test kDFS with VMs vmware images qemu images

[edit] kernel/kerrighed Distributed File System

kDFS aims at providing an integrated cluster file system for High Performance Computing.

Based on several concepts suggested in KerFS (Kerrighed 1.02), the kernel Distributed File system has been develop from scratch. One of the main idea consists in developing a distributed file system pluggable under the VFS and only based on the KDDM component of Kerrighed. The KDDM features are used to build a cooperative cache for both data and meta-data using all available memory in the cluster.

• Further informations is available in the following pdf presentation:

kDFS Overview: current state and main objectives (june 2007)

kDFS, Alpha version (october 2007)

A research report addressing design and current implementation of kDFS is available there (december 2007). Feel free to contact us if you are interested by a longer version or any further information.

Contact : Adrien Lebre / Pierre Riteau / Marko Obrovac

[edit] Developers

[edit] Adrien Lèbre

Debugging/Finalizing the current code. Working on I/O scheduler strategies and dynamical RAID placement. Since september 2008, my activities have been focusing on the integration issues between kDFS and the other cluster services (by exploiting aspect oriented programming approaches).

[edit] Pierre Riteau

During my Computer Science Master, I worked on adding file checkpointing capabilities to kDFS, using Copy-On-Write and data distribution over different nodes. In mid-term, this will be used by the checkpoint/restart services of Kerrighed to improve fault tolerance. In a longer term, such features should be available whatever the file system. During summer 2008 I worked on kDFS to port it to the kddm-standalone framework, debug/finalize the code, add diskless support, etc.

[edit] Marko Obrovac

Currently I'm expanding the kDFS superblock management. At the same time, I'm studying several disk models. Based on this analysis, several I/O probes will be implemented as part of kDFS in order to provide better global load-balancing scheduling. This approach should help the scheduler to improve global cluster performances. This work is done as part of my master internship. I hope that such an approach could be integrated in any future cluster-aware file system.

[edit] Latest news

[edit] November 2008: kDFS at SuperComputing 2008 - booth 2203

Pierre will present a complete demo of kDFS and will describe the latest kDFS developments. Don't hesitate to visit the INRIA booth.

[edit] August 2008: kDFS is available as a standalone kernel module

kDFS was recently ported to work on the kDDM standalone framework. This means kDFS can now be used without Kerrighed. More information is available by downloading the kdfs-standalone branch in the Kerrighed repository: svn checkout svn://scm.gforge.inria.fr/svn/kerrighed/branches/kdfs-standalone

[edit] May 1 2008: kDFS design will be presented at the Europar Conference

The paper "Reducing Development Complexity in Distributed Environments" has been accepted to the Europar Conference 2008. This paper introduces the design of kDFS based on the kDDM layer. This paper will be officially available by the end of august 2008. In the meantime, you can have a look at our internal research report !

[edit] Apr 29 2008: Looking for a new contributor

We are looking for a new contributor to port kDFS from the Kerrighed framework to the kDDM standalone module under Linux 2.6.25. Even if the main developments will still focus on the kDFS integration within Kerrighed, we would like to provide kDFS as an independent module. C programmers are welcome, don't hesitate to contact us !

[edit] Apr 28 2008: kDFS passes the NTFS-3G POSIX File System Test Suite

kDFS is able to successfully complete the NTFS-3G POSIX File System Test Suite.

All tests successful.
Files=184, Tests=1950, 161 wallclock secs ( 9.47 cusr + 82.34 csys = 91.81 CPU)

[edit] Virtual machines

If you want to play with kDFS, we provide fully configured VMWare & QEMU virtual machines. The first virtual machine is a DHCP/NFS server that allows to netboot four other virtual machines. These four virtual machines can be used to create Kerrighed clusters running kDFS.

The IPs 192.168.0.1, 192.168.0.2, 192.168.0.3, 192.168.0.4 and 192.168.0.16 are used by the virtual machines, be careful if you have the same IPs in your network.

WARNING:

The QEMU virtual machines are rather slow, so you're encouraged to play with the VMWare machines.

The kDFS code provided in the different VMs is outdated (feb 2008), please check the README file to update it from our SVN!

[edit] VMWare virtual machines

You can use these virtual machines with VMWare Fusion on Mac OS X. On Linux you can use VMWare Player. Please note that the given images do not run with VMWare 1.0.4 (VMWare 2 is required). Before starting the VMWare virtual machines, please read this readme file.

Links to the files:

• KerrighedServer.tar.bz2 (WARNING, this file's size is 890 MB)
• LinuxKrg1.tar.bz2
• LinuxKrg2.tar.bz2
• LinuxKrg3.tar.bz2
• LinuxKrg4.tar.bz2

[edit] QEMU virtual machines

In order to run the QEMU virtual machines, you need to get QEMU. There's also a Mac OS X port, but you should be aware that it hasn't been tested. If you successfully ran all the kDFS configuration on Mac OS X, please let us know. Before starting the QEMU virtual machines, please read this readme file.

Links to the files:

• KerrighedServer.tar.gz (WARNING, this file's size is 1 GB)
• LinuxKrg1.tar.gz
• LinuxKrg2.tar.gz
• LinuxKrg3.tar.gz
• LinuxKrg4.tar.gz