coMosix Installation

Etienne Bagnoud

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled " Appendix A, GNU Free Documentation License".

Revision History
Revision 1.32006-07-03sb
Grammatical and orthographic corrections.
Revision 1.22006-06-28eb
Creation of the DocBook version. Addition and correction of some informations.
Revision 1.12006-06-12eb
First HTML version from the README file.
Revision 1.02006-06-07eb
First release as a README file for coMosix project.

openMosix on Windows

coMosix is the name of coLinux and openMosix contracted. It's a project that aims to make openMosix running on Windows.

The project started somewhere in March 2005. The first known date is Mar 29, 2005. This was the date of the first tools written for that project (see Section 10.1, “Source of net_card_name.exe”).

Run an openMosix cluster under Windows and Linux. coMosix is the mix of coLinux, openMosix and some other Linux hack to create an easy to use clustering solution. coMosix provides a simple (silent-)installer that can be thrown over a Windows Network.


Table of Contents

1. About this document
1.1. About this document
1.2. Feedback
1.3. Acknowledgments and Thanks
2. Install coMosix
2.1. NFS Share
2.2. DHCP Server
2.3. WinPcap
2.4. comosix.ini
2.4.1. coMosix section
2.4.2. NFS section
2.4.3. cobd section
2.5. Sample procedure to install coMosix
2.5.1. Preparing the NFS Server
2.5.2. Configure coMosix for Windows install
2.5.2.1. Configuration of comosix section
2.5.2.2. Configuration of NFS section
2.5.2.3. Configuration of cobd section
3. Default root password
4. Root file system
4.1. Files linked to /tmp/comosix
5. Dynamic DNS
6. Others changes in root file system
7. WinPcap
8. Used configuration
9. Compile your own installer
9.1. Compile your installer on Windows or Linux ?
9.2. Installer source
9.2.1. Source structure
9.2.2. Compile coLinux kernel
9.2.2.1. The make modules_install problem and System.map
10. net_card_name.exe
10.1. Source of net_card_name.exe
11. Troubleshooting
11.1. Windows XP SP2
11.2. DynamicDNS and DHCP
Bibliography
A. GNU Free Documentation License
1. PREAMBLE
2. APPLICABILITY AND DEFINITIONS
3. VERBATIM COPYING
4. COPYING IN QUANTITY
5. MODIFICATIONS
6. COMBINING DOCUMENTS
7. COLLECTIONS OF DOCUMENTS
8. AGGREGATION WITH INDEPENDENT WORKS
9. TRANSLATION
10. TERMINATION
11. FUTURE REVISIONS OF THIS LICENSE
12. ADDENDUM: How to use this License for your documents

1. About this document

1.1. About this document

This document was started on Jun 7, 2006 by Etienne Bagnoud as a README file for coMosix project.

1.2. Feedback

Send feedback to < comosix-documentation@lists.sourceforge.net>. You must subscribe in order to send email to the list.

1.3. Acknowledgments and Thanks

Thanks to all the authors of the LDP Author Guide for their document.

2. Install coMosix

In order to have coMosix working on your system, you'll need the root file system available as an NFS share, then you'll have to configure your installer with an ini file. When you've done that. You can run the installer on every Windows station of your network.

Don't forget that you'll need a working DHCP server in order to provide an IP address to the Linux kernel.

2.1. NFS Share

To install coMosix, you'll have to first setup your NFS share with omrootfs.tar.bz2. It can be found on sourceforge at http://prdownloads.sourceforge.net/comosix/ omrootfs.tar.bz2?download. This is a complete debian installation ready to use with coMosix. You can build your own installation if you want, but this should work.

To use the provided root file system, just decompress omrootfs.tar.bz2 somewhere on your NFS server, and share it. More info on NFS Server can be found here http://nfs.sourceforge.net/nfs-howto/server.html

2.2. DHCP Server

coMosix need a DHCP Server, to setup one go there : http://www.tldp.org/HOWTO/DHCP/x369.html

2.3. WinPcap

In order to work, coMosix needs WinPcap installed on the Windows station you want to use as a cluster node. You can get WinPcap here : http://www.winpcap.org/.

2.4. comosix.ini

Then you can configure your installer. What appears with coMosix 1.2.1 is that it can be fully configured by its ini file. The first limitation is that you have to keep the name comosix.ini and it must be in the ExeDir of the installer.

In comosix.ini you'll find three sections. The coMosix section, the NFS section and the cobd section.

2.4.1. coMosix section

It contains the memory size used by coLinux. If not set, the default value is 16Mo. If you want to set more, just add memory_size=X, where X is the value in Mo that you want. For example :

Example 1. comosix section in comosix.ini

[comosix]
memory_size=64

2.4.2. NFS section

It contains three values. You have server that contains the IP address of the NFS server, share that contains the path of the share on the NFS server and finally options that contains NFS options with coma-separated value.

By default the values are server=192.168.1.1, path=/comosix and no options set.

For example at IRO, we use the following NFS configuration :

Example 2. IP Configuration for NFS section

[nfs]
server=192.168.1.22
share=/data/comosix/omrootfs

To find more informations about root file system mounted with NFS, you can have a look here : http://www.vdberg.org/~richard/Linux-on-SunPCi-mini-Howto/nfsroot .txt.

2.4.3. cobd section

cobd means coLinux Block Device.

The cobd section allow you to setup some coLinux Block Device ( ten exactly). If you want to setup three cobd, do just like that :

Example 3. Cobd example

[cobd]
path_0=\DosDevice\C:\my_first_cobd
path_1=\DosDevice\C:\my_second_cobd
path_2=\DosDevice\C:\my_third_cobd

Under coLinux you can access it like that : /dev/cobd0 -> C:\my_first_cobd , /dev/cobd1 -> C:\my_first_cobd and /dev/cobd2 -> C:\my_third_cobd. There's a limitation here. You cannot jump over a number, for example path_0, path_2 without path_1. In that case the configuration file will just contains path_0, others will be sliped. You can find more information here : http://wiki.colinux.org/mediawiki/index.php/Block_device

If you've setup some cobds, you would surely mount them into you coLinux. As coMosix is designed to run with a Read Only NFS share , some files exist just in RAM disk. So in order to use /etc/fstab to describe your mount points for your cobd, you must use /etc/fstab.local in your rootfs (in case you've build your own file system, you may do it in another way. But if you use the provided rootfs, do as said !)

Now you're ready to install coMosix. Just keep the comosix.ini in the same directory as the installer and your coMosix installation can run.

If you want to build your own coLinux kernel, you'll have to compile your own installer. As all the files are provided with coMosix project, just read the section Section 9, “Compile your own installer” in this file.

2.5. Sample procedure to install coMosix

We are going to install coMosix on the following network :

  • Debian server with NFS installed named embryo
  • Client Windows XP SP2 named marooned

Warning

The configuration of DHCP server is skipped here. I assume that you already have a DHCP server running on your network. The DNS configuration is also skipped but if you use DynamicDNS assignment, I strongly recommend to change the file /etc/init.d/hostname.sh in the root file system.

2.5.1. Preparing the NFS Server

First thing we are going to do is to setup the NFS Server. Assuming you already have a running NFS Server on your system.

For this example I'm working in the directory /var/comosix on embryo

Let's download omrootfs.tar.bz2 from sourceforge.net and unpack it :

Warning

omrootfs.tar.bz2 is about 86 Mo

Example 4. Download omrootfs.tar.bz2

tchetch@embryo:/var/comosix$ wget http://switch.dl.sourceforge.net/sourceforge/comosix/omrootfs.tar.bz2
tchetch@embryo:/var/comosix$ tar xjvf omrootfs.tar.bz2

You might want to change the default root password of the provided rootfs you can easily do it that way (see Section 3, “Default root password” for the default root password), first switch to root in order to chroot.

By the way we are already going to change a little thing that can confuse you or your network. The problem is that the hostname send to the DHCP Server is the same as the Windows machine (see Section 11.2, “DynamicDNS and DHCP” for more info) :

Example 5. Changing the default password

tchetch@embryo:/var/comosix$ su
embryo:/var/comosix# chroot omrootfs
root@embryo:/# passwd
root@embryo:/# vi /etc/init.d/hostname.sh

Now in /etc/init.d/hostname.sh change the line echo "send host-name \""`hostname`"\";" > /etc/dhclient.conf into : echo "send host-name \"prefix_"`hostname`"\";" > /etc/dhclient.conf where prefix can be anything you want (I use "om_" for openMosix).

As you are working in the chrooted environment, you can do almost everything, for example updating your Debian system. But as you are working on system that is designed to be ReadOnly, before accessing to an extern server, you would need to tell where is the DNS server in your environment :

Example 6. Creating file to work on Internet

root@embryo:/# mkdir /tmp/comosix
root@embryo:/# echo "nameserver 192.168.1.22" > /etc/resolv.conf

Creating the directory /tmp/comosix allows you to create files that are normally created at coMosix boot time. But when working in chrooted environment we may need those files, see Section 4.1, “Files linked to /tmp/comosix.

Now you can surf on the Web as you want, why not upgrading the whole Debian distribution ? So let's go :

Example 7. Upgrading Debian

root@embryo:/# apt-get update
root@embryo:/# apt-get upgrade

Voilà ! It's done. As you can see, there's a lot of possibility while working in the chrooted environment.

And now you can share /var/comosix/omrootfs as ReadOnly share. Just edit /etc/exports and add the following line : /var/comosix/omrootfs *(ro,sync,all_squash) and reload NFS, but don't forget to exit chrooted environment :

Example 8. Configure NFS

root@embryo:/# exit
embryo:/var/comosix# echo "/var/comosix/omrootfs *(ro,sync,all_squash) >> /etc/exports
embryo:/var/comosix# /etc/init.d/nfs-user-server reload

Having done that you are almost ready to install coMosix. If you want more control on NFS options in /etc/exports, take a good documentation about that.

2.5.2. Configure coMosix for Windows install

Warning

If you have Windows XP SP2, read Section 11.1, “Windows XP SP2”

Now we have a working NFS share, so configuration of the coMosix installer will be very quick :

Download on your Windows station the coMosix-1.2.1-Sinstall.exe from http://prdownloads.sourceforge.net/comosix/coMosix-1.2.1-Sinstall.exe?download and put it into a folder, for example C:\comosix_test\.

2.5.2.1. Configuration of comosix section

First we set the memory we will allocate to our coMosix on Windows. This is the only value found in the "comosix" section in comosix.ini. Choose how much memory you want to allow to coMosix on Windows and then set it, for me 64 Mo is good :

Example 9. Set memory

C:\>echo [comosix] > C:\comosix_test\comosix.ini
C:\>echo memory_size=64 >> C:\comosix_test\comosix.ini

2.5.2.2. Configuration of NFS section

Then we'll set the NFS section with its parameters. My NFS Server is found at IP 192.168.1.114 and my path is /var/comosix/omrootfs, so :

Example 10. Set NFS

C:\>echo [NFS] >> C:\comosix_test\comosix.ini
C:\>echo server=192.168.1.114 >> C:\comosix_test\comosix.ini
C:\>echo path=/var/comosix/omrootfs >> C:\comosix_test\comosix.ini

2.5.2.3. Configuration of cobd section

I don't set any cobd file, so :

Example 11. Set cobd

C:\>echo [cobd] >> C:\comosix_test\comosix.ini

Well now run the coMosix-1.2.1-Sinstall.exe and see if your coMosix works. You can have access to it with ssh.

3. Default root password

On the distributed root file system, the default root password is comosix.

4. Root file system

You can find a ready to work here : http://prdownloads.sourceforge.net/comosix/omrootfs.tar.bz2?download .

The root file system is based on Debian. You can use apt-get or most of the usual Debian stuff.

Some files are created at boot time in a ram disk device, like /etc/fstab. The script comosix in /etc/init.d/ is the first script called by init. It build all the necessary files and mount all the necessary ram disk device for /var, /tmp, /root and /home. No file would be kept on system restart.

You can configure other mounting point by using /etc/fstab.local, which is used by comosix init script to build the /etc/fstab. The rest of the init process is the usual one.

SSH key are generated in the comosix init script. It's a temporary solution for that, because each time you log on a coLinux via ssh, it complains that keys have changed.

To work on the root file system, you can use chroot.

4.1. Files linked to /tmp/comosix

Some files needs to be set as writable in coMosix. For those files the solution is to create Ram Disk Device and create those files in it. So /var , /tmp, /root and /home are mounted in a Ram Disk Device.

Then we link some files in /etc to /tmp/comosix in order to make them writable :

  • /etc/dhclient.conf
  • /etc/fstab
  • /etc/motd
  • /etc/motd.tmp
  • /etc/nologin
  • /etc/resolv.conf
  • /etc/network/ifstate

By doing this we have a ReadOnly root file system, but still some files are writable

5. Dynamic DNS

You can use Dynamic DNS for your cluster. At boot time, the coLinux nodes send the windows machine name with send host-name.

6. Others changes in root file system

Some little changes have been made like removing rootfs check in the init script and changing the hostname in the init script was changed to use the windows machine name. The network init script was changed to force the re-setup via DHCP of the network interface to use dynamic DNS, because the DHCP configuration made by the kernel don't allow us to send what we want.

7. WinPcap

coMosix needs WinPcap installed before the installation of coMosix. Get WinPcap from http://www.winpcap.org/.

8. Used configuration

For OpenMosix, we use this configuration :

Example 12. openMosix configuration

[*] openMosix process migration support
[ ] Support clusters with a complex network topology
[*] Stricter security on openMosix ports
(1) Level of process-identity disclosure (0-3)
[*] Poll/Select exceptions on pipes
[*] Disable OOM Killer
[ ] Shared memory migration support (Experimental)
[*] Enable Extension: Local Time

For the rest, the coLinux IP configuration is done via DHCP. It also requires to set the nfsroot in the file comosix.ini to fit your NFS server configuration.

9. Compile your own installer

To compile your own installer, you'll need Nullsoft Scriptable Install System, get it from http://nsis.sourceforge.net/Main_Page.

9.1. Compile your installer on Windows or Linux ?

You can compile your installer on both system. I have been using Linux for sometimes in order to compile the Windows installer, and now I'm using Windows for that job.

After having used both, I would say that Windows fits better to do this work because you don't need to copy the compiled installer onto a Windows station to test.

If you want to use Linux, I have compiled using NSIS with Wine. It works very well. You can find Wine at http://www.winehq.com/.

9.2. Installer source

In order to compile your installer, you need the source files provided by the coMosix project. You can get the sources from http://prdownloads.sourceforge.net/comosix/comosix-installer-src-1.2.1.tar.bz2?download .

Unpack the file, using tar under Linux or 7zip under Windows. Now you have all the needed sources to prepare you installer.

9.2.1. Source structure

Now that the sources are unpacked, just go to the sources root , for example C:\comosix-installer-src-1.2.1\ or /home/tchetch/comosix-installer-src-1.2.1 , and you will see a directory called ./nsis which contains the installer source file and some other files :

Files in installer sources

  • CHANGELOG
  • LICENSE
  • README
  • colinux-bridged-net-daemon.exe
  • colinux-daemon.exe
  • comosix.xml
  • linux.sys
  • net_card_name.exe
  • vmlinux

The only file that you would really change is vmlinux. The rest are coLinux files that are set by comosix.ini at installation, see Section 2.4, “comosix.ini”.

9.2.2. Compile coLinux kernel

Warning

This job will be done under a Linux system. I've tried to compile it under Windows but it did not work.

This is the Linux kernel for coLinux. You may change this file in order to adapt coMosix to your already build openMosix cluster. This file is generated from the coLinux patched sources. You can download it from http://prdownloads.sourceforge.net/comosix/colinux-kernel.tar.bz2?download .

Compiling coLinux kernel is similar as a "standard" Linux, so I will skip details on it, you would find enough documentation about that on the Web

What you will find inside the tarball is a Linux source already patched to run with coMosix and ./CONFIG-r1. This file is the actual configuration file of coMosix. Clean the source tree, with make mrproper, copy ./CONFIG-r1 to ./.config, make menuconfig to configure the kernel, change openMosix settings as you want and make dep bzImage modules.

Now you have ./vmlinux, you can copy this in order to replace $INSTALLER_SRC/vmlinux.

9.2.2.1. The make modules_install problem and System.map

The only things that you would do is to install modules with make modules_install. The problem is that you have compiled coLinux kernel and the modules must be installed in rootfs. The rootfs is an NFS share, so you must mount this share as ReadWrite, for example /mnt/comosix_rootfs.

Then you are ready to install your modules, the command line is easy : INSTALL_MODE_PATH=/mnt/comosix_rootfs make modules_install . And copy the System.map to /mnt/comosix_rootfs/boot/System.map.

Now your new coLinux kernel is ready to work. You can start building your installer.

To compile your installer, you just have to run the .nsis file corresponding to your coMosix version into the NSIS compiler.

10. net_card_name.exe

This program has been made to get the right network card name under Windows. I've seen a really funny error while trying to get that name. Indeed French version of Windows doesn't give the right card name (right=usable for WinPcap). So I created this little soft that ask the listing of the card in the computer but it uses WinPcap to ask the net card name and so it get the right name.

10.1. Source of net_card_name.exe

This is the code I wrote more than a year ago in order to get the right card name under a french version of Windows XP/2000.

Example 13. Source code of net_card_name.exe

		
/* 
 * AUTHOR : Etienne Bagnoud <tchetch@i-james.com>
 * DATE : 2005/03/29
 * 
 * This program retrieve the name of the network card
 */
#include <stdio.h>;
#include <stdlib.h>;
#include <string.h>;
#include <pcap.h>;

int main(int argc, char **argv)
{
	pcap_if_t *alldevs;
	pcap_if_t *d;
	int i=0;
	int found = 0;
	char errbuf[PCAP_ERRBUF_SIZE];
	FILE *fp;
	FILE *fp1;
	
	fp1 = fopen("error_net_card_name.log", "w");
	/* Retrieve the device list */

	if (pcap_findalldevs(&alldevs, errbuf) == -1)
	{
		fprintf(fp1 ,"Error in pcap_findalldevs: %s\n", errbuf);
		fclose(fp1);
		exit(-1);
	}
	/* Print the list */

	for(d=alldevs;d;d=d->next)
	{
		if (d->description)
		{
			if(!(strcmp(d->description, "Generic NdisWan adapter") == 0))
			{
				found = 1;
				break;
			}
		}
		i++;
	}
	
	if(i==0 || found == 0)
	{
		pcap_freealldevs(alldevs);
		fprintf(fp1, "pcap_freealldevs(alldevs); (0)!!!\n");
		fclose(fp1);
		exit(-1);
	}
	
	if(found == 1)
	{
		if(!(fp = fopen("winpcap_card_name", "w")))
		{
			pcap_freealldevs(alldevs);
			fprintf(fp1, "pcap_freealldevs(alldevs); (1)!!!\n");
			fclose(fp1);
			exit(-1);
		}
		fprintf(fp, "%s", d->description);
	}
	
	/* We don't need any more the device list. Free it */
	pcap_freealldevs(alldevs);
	fclose(fp1);
	return 0;
}

11. Troubleshooting

11.1. Windows XP SP2

If you've got a Windows XP SP2 with new processor (that support NoExecute), you should add /noexecute=AlwaysOff in your boot.ini in case you don't do that, Windows will crash at every boot.

More info on the coLinux Wiki : http://wiki.colinux.org/mediawiki/index.php/Getting_Started_with_coLinux_-_Long_manual#coLinux_0.6.1_and_XP_SP2_with_PAE

11.2. DynamicDNS and DHCP

The DHCP client can send information to the DHCP server. For DynamicDNS, the DHCP server uses the hostname send by the client to set the DNS name.

The problem is that coMosix send the same hostname as Windows and thus creates a collision. Of course, the DHCP server will not use two times the same DNS entry for two different machines, but it will discard one or give another name.

So in order to have coMosix and Windows having their own DNS name, you can modify the file /etc/init.d/hostname.sh as explained in Section 2.5, “Sample procedure to install coMosix”.

Bibliography

openMosix. http://www.openmosix.org/.

openMosix is a Linux kernel extension for single-system image clustering which turns a network of ordinary computers into a supercomputer.

coLinux. http://www.colinux.org/.

Cooperative Linux is the first working free and open source method for optimally running Linux on Microsoft Windows natively.

Build a heterogeneous cluster with coLinux and openMosix. http://www-128.ibm.com/developerworks/linux/library/l-colinux/ .

An experimental case study combines coLinux with an openMosix-enabled kernel to build a hybrid cluster
Mulyadi Santosa and Andreas Schaefer.

The Harpy Project. http://cluster.inf-ra.uni-jena.de/harpy .

We'd like to dynamically enlarge our cluster by adding under-utilized Windows hosts on the fly. Therefore we boot an coLinux VM within the running Windows and migrate processes from the cluster onto the Windows machine via openMosix.

A. GNU Free Documentation License

1. PREAMBLE

The purpose of this License is to make a manual, textbook, or other functional and useful document "free" in the sense of freedom: to assure everyone the effective freedom to copy and redistribute it, with or without modifying it, either commercially or noncommercially. Secondarily, this License preserves for the author and publisher a way to get credit for their work, while not being considered responsible for modifications made by others.

This License is a kind of "copyleft", which means that derivative works of the document must themselves be free in the same sense. It complements the GNU General Public License, which is a copyleft license designed for free software.

We have designed this License in order to use it for manuals for free software, because free software needs free documentation: a free program should come with manuals providing the same freedoms that the software does. But this License is not limited to software manuals; it can be used for any textual work, regardless of subject matter or whether it is published as a printed book. We recommend this License principally for works whose purpose is instruction or reference.

2. APPLICABILITY AND DEFINITIONS

This License applies to any manual or other work, in any medium, that contains a notice placed by the copyright holder saying it can be distributed under the terms of this License. Such a notice grants a world-wide, royalty-free license, unlimited in duration, to use that work under the conditions stated herein. The "Document", below, refers to any such manual or work. Any member of the public is a licensee, and is addressed as "you". You accept the license if you copy, modify or distribute the work in a way requiring permission under copyright law.

A "Modified Version" of the Document means any work containing the Document or a portion of it, either copied verbatim, or with modifications and/or translated into another language.

A "Secondary Section" is a named appendix or a front-matter section of the Document that deals exclusively with the relationship of the publishers or authors of the Document to the Document's overall subject (or to related matters) and contains nothing that could fall directly within that overall subject. (Thus, if the Document is in part a textbook of mathematics, a Secondary Section may not explain any mathematics.) The relationship could be a matter of historical connection with the subject or with related matters, or of legal, commercial, philosophical, ethical or political position regarding them.

The "Invariant Sections" are certain Secondary Sections whose titles are designated, as being those of Invariant Sections, in the notice that says that the Document is released under this License. If a section does not fit the above definition of Secondary then it is not allowed to be designated as Invariant. The Document may contain zero Invariant Sections. If the Document does not identify any Invariant Sections then there are none.

The "Cover Texts" are certain short passages of text that are listed, as Front-Cover Texts or Back-Cover Texts, in the notice that says that the Document is released under this License. A Front-Cover Text may be at most 5 words, and a Back-Cover Text may be at most 25 words.

A "Transparent" copy of the Document means a machine-readable copy, represented in a format whose specification is available to the general public, that is suitable for revising the document straightforwardly with generic text editors or (for images composed of pixels) generic paint programs or (for drawings) some widely available drawing editor, and that is suitable for input to text formatters or for automatic translation to a variety of formats suitable for input to text formatters. A copy made in an otherwise Transparent file format whose markup, or absence of markup, has been arranged to thwart or discourage subsequent modification by readers is not Transparent. An image format is not Transparent if used for any substantial amount of text. A copy that is not "Transparent" is called "Opaque".

Examples of suitable formats for Transparent copies include plain ASCII without markup, Texinfo input format, LaTeX input format, SGML or XML using a publicly available DTD, and standard-conforming simple HTML, PostScript or PDF designed for human modification. Examples of transparent image formats include PNG, XCF and JPG. Opaque formats include proprietary formats that can be read and edited only by proprietary word processors, SGML or XML for which the DTD and/or processing tools are not generally available, and the machine-generated HTML, PostScript or PDF produced by some word processors for output purposes only.

The "Title Page" means, for a printed book, the title page itself, plus such following pages as are needed to hold, legibly, the material this License requires to appear in the title page. For works in formats which do not have any title page as such, "Title Page" means the text near the most prominent appearance of the work's title, preceding the beginning of the body of the text.

A section "Entitled XYZ" means a named subunit of the Document whose title either is precisely XYZ or contains XYZ in parentheses following text that translates XYZ in another language. (Here XYZ stands for a specific section name mentioned below, such as "Acknowledgements", "Dedications", "Endorsements", or "History".) To "Preserve the Title" of such a section when you modify the Document means that it remains a section "Entitled XYZ" according to this definition.

The Document may include Warranty Disclaimers next to the notice which states that this License applies to the Document. These Warranty Disclaimers are considered to be included by reference in this License, but only as regards disclaiming warranties: any other implication that these Warranty Disclaimers may have is void and has no effect on the meaning of this License.

3. VERBATIM COPYING

You may copy and distribute the Document in any medium, either commercially or noncommercially, provided that this License, the copyright notices, and the license notice saying this License applies to the Document are reproduced in all copies, and that you add no other conditions whatsoever to those of this License. You may not use technical measures to obstruct or control the reading or further copying of the copies you make or distribute. However, you may accept compensation in exchange for copies. If you distribute a large enough number of copies you must also follow the conditions in section 3.

You may also lend copies, under the same conditions stated above, and you may publicly display copies.

4. COPYING IN QUANTITY

If you publish printed copies (or copies in media that commonly have printed covers) of the Document, numbering more than 100, and the Document's license notice requires Cover Texts, you must enclose the copies in covers that carry, clearly and legibly, all these Cover Texts: Front-Cover Texts on the front cover, and Back-Cover Texts on the back cover. Both covers must also clearly and legibly identify you as the publisher of these copies. The front cover must present the full title with all words of the title equally prominent and visible. You may add other material on the covers in addition. Copying with changes limited to the covers, as long as they preserve the title of the Document and satisfy these conditions, can be treated as verbatim copying in other respects.

If the required texts for either cover are too voluminous to fit legibly, you should put the first ones listed (as many as fit reasonably) on the actual cover, and continue the rest onto adjacent pages.

If you publish or distribute Opaque copies of the Document numbering more than 100, you must either include a machine-readable Transparent copy along with each Opaque copy, or state in or with each Opaque copy a computer-network location from which the general network-using public has access to download using public-standard network protocols a complete Transparent copy of the Document, free of added material. If you use the latter option, you must take reasonably prudent steps, when you begin distribution of Opaque copies in quantity, to ensure that this Transparent copy will remain thus accessible at the stated location until at least one year after the last time you distribute an Opaque copy (directly or through your agents or retailers) of that edition to the public.

It is requested, but not required, that you contact the authors of the Document well before redistributing any large number of copies, to give them a chance to provide you with an updated version of the Document.

5. MODIFICATIONS

You may copy and distribute a Modified Version of the Document under the conditions of sections 2 and 3 above, provided that you release the Modified Version under precisely this License, with the Modified Version filling the role of the Document, thus licensing distribution and modification of the Modified Version to whoever possesses a copy of it. In addition, you must do these things in the Modified Version:

GNU FDL Modification Conditions

  1. Use in the Title Page (and on the covers, if any) a title distinct from that of the Document, and from those of previous versions (which should, if there were any, be listed in the History section of the Document). You may use the same title as a previous version if the original publisher of that version gives permission.
  2. List on the Title Page, as authors, one or more persons or entities responsible for authorship of the modifications in the Modified Version, together with at least five of the principal authors of the Document (all of its principal authors, if it has fewer than five), unless they release you from this requirement.
  3. State on the Title page the name of the publisher of the Modified Version, as the publisher.
  4. Preserve all the copyright notices of the Document.
  5. Add an appropriate copyright notice for your modifications adjacent to the other copyright notices.
  6. Include, immediately after the copyright notices, a license notice giving the public permission to use the Modified Version under the terms of this License, in the form shown in the Addendum below.
  7. Preserve in that license notice the full lists of Invariant Sections and required Cover Texts given in the Document's license notice.
  8. Include an unaltered copy of this License.
  9. Preserve the section Entitled "History", Preserve its Title, and add to it an item stating at least the title, year, new authors, and publisher of the Modified Version as given on the Title Page. If there is no section Entitled "History" in the Document, create one stating the title, year, authors, and publisher of the Document as given on its Title Page, then add an item describing the Modified Version as stated in the previous sentence.
  10. Preserve the network location, if any, given in the Document for public access to a Transparent copy of the Document, and likewise the network locations given in the Document for previous versions it was based on. These may be placed in the "History" section. You may omit a network location for a work that was published at least four years before the Document itself, or if the original publisher of the version it refers to gives permission.
  11. For any section Entitled "Acknowledgements" or "Dedications", Preserve the Title of the section, and preserve in the section all the substance and tone of each of the contributor acknowledgements and/or dedications given therein.
  12. Preserve all the Invariant Sections of the Document, unaltered in their text and in their titles. Section numbers or the equivalent are not considered part of the section titles.
  13. Delete any section Entitled "Endorsements". Such a section may not be included in the Modified Version.
  14. Do not retitle any existing section to be Entitled "Endorsements" or to conflict in title with any Invariant Section.
  15. Preserve any Warranty Disclaimers.

If the Modified Version includes new front-matter sections or appendices that qualify as Secondary Sections and contain no material copied from the Document, you may at your option designate some or all of these sections as invariant. To do this, add their titles to the list of Invariant Sections in the Modified Version's license notice. These titles must be distinct from any other section titles.

You may add a section Entitled "Endorsements", provided it contains nothing but endorsements of your Modified Version by various parties--for example, statements of peer review or that the text has been approved by an organization as the authoritative definition of a standard.

You may add a passage of up to five words as a Front-Cover Text, and a passage of up to 25 words as a Back-Cover Text, to the end of the list of Cover Texts in the Modified Version. Only one passage of Front-Cover Text and one of Back-Cover Text may be added by (or through arrangements made by) any one entity. If the Document already includes a cover text for the same cover, previously added by you or by arrangement made by the same entity you are acting on behalf of, you may not add another; but you may replace the old one, on explicit permission from the previous publisher that added the old one.

The author(s) and publisher(s) of the Document do not by this License give permission to use their names for publicity for or to assert or imply endorsement of any Modified Version.

6. COMBINING DOCUMENTS

You may combine the Document with other documents released under this License, under the terms defined in section 4 above for modified versions, provided that you include in the combination all of the Invariant Sections of all of the original documents, unmodified, and list them all as Invariant Sections of your combined work in its license notice, and that you preserve all their Warranty Disclaimers.

The combined work need only contain one copy of this License, and multiple identical Invariant Sections may be replaced with a single copy. If there are multiple Invariant Sections with the same name but different contents, make the title of each such section unique by adding at the end of it, in parentheses, the name of the original author or publisher of that section if known, or else a unique number. Make the same adjustment to the section titles in the list of Invariant Sections in the license notice of the combined work.

In the combination, you must combine any sections Entitled "History" in the various original documents, forming one section Entitled "History"; likewise combine any sections Entitled "Acknowledgements", and any sections Entitled "Dedications". You must delete all sections Entitled "Endorsements".

7. COLLECTIONS OF DOCUMENTS

You may make a collection consisting of the Document and other documents released under this License, and replace the individual copies of this License in the various documents with a single copy that is included in the collection, provided that you follow the rules of this License for verbatim copying of each of the documents in all other respects.

You may extract a single document from such a collection, and distribute it individually under this License, provided you insert a copy of this License into the extracted document, and follow this License in all other respects regarding verbatim copying of that document.

8. AGGREGATION WITH INDEPENDENT WORKS

A compilation of the Document or its derivatives with other separate and independent documents or works, in or on a volume of a storage or distribution medium, is called an "aggregate" if the copyright resulting from the compilation is not used to limit the legal rights of the compilation's users beyond what the individual works permit. When the Document is included in an aggregate, this License does not apply to the other works in the aggregate which are not themselves derivative works of the Document.

If the Cover Text requirement of section 3 is applicable to these copies of the Document, then if the Document is less than one half of the entire aggregate, the Document's Cover Texts may be placed on covers that bracket the Document within the aggregate, or the electronic equivalent of covers if the Document is in electronic form. Otherwise they must appear on printed covers that bracket the whole aggregate.

9. TRANSLATION

Translation is considered a kind of modification, so you may distribute translations of the Document under the terms of section 4. Replacing Invariant Sections with translations requires special permission from their copyright holders, but you may include translations of some or all Invariant Sections in addition to the original versions of these Invariant Sections. You may include a translation of this License, and all the license notices in the Document, and any Warranty Disclaimers, provided that you also include the original English version of this License and the original versions of those notices and disclaimers. In case of a disagreement between the translation and the original version of this License or a notice or disclaimer, the original version will prevail.

If a section in the Document is Entitled "Acknowledgements", "Dedications", or "History", the requirement (section 4) to Preserve its Title (section 1) will typically require changing the actual title.

10. TERMINATION

You may not copy, modify, sublicense, or distribute the Document except as expressly provided for under this License. Any other attempt to copy, modify, sublicense or distribute the Document is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.

11. FUTURE REVISIONS OF THIS LICENSE

The Free Software Foundation may publish new, revised versions of the GNU Free Documentation License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns. See http://www.gnu.org/copyleft/.

Each version of the License is given a distinguishing version number. If the Document specifies that a particular numbered version of this License "or any later version" applies to it, you have the option of following the terms and conditions either of that specified version or of any later version that has been published (not as a draft) by the Free Software Foundation. If the Document does not specify a version number of this License, you may choose any version ever published (not as a draft) by the Free Software Foundation.

12. ADDENDUM: How to use this License for your documents

To use this License in a document you have written, include a copy of the License in the document and put the following copyright and license notices just after the title page:

Sample Invariant Sections list

Copyright (c) YEAR YOUR NAME. Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled "GNU Free Documentation License".

If you have Invariant Sections, Front-Cover Texts and Back-Cover Texts, replace the "with...Texts." line with this:

Sample Invariant Sections list

with the Invariant Sections being LIST THEIR TITLES, with the Front-Cover Texts being LIST, and with the Back-Cover Texts being LIST.

If you have Invariant Sections without Cover Texts, or some other combination of the three, merge those two alternatives to suit the situation.

If your document contains nontrivial examples of program code, we recommend releasing these examples in parallel under your choice of free software license, such as the GNU General Public License, to permit their use in free software.