Ci-dessous, les différences entre deux révisions de la page.
| Les deux révisions précédentesRévision précédenteProchaine révision | Révision précédente | ||
| en:developpement:productions:sidus [2013/11/06 20:55] – [Startup sequence] equemene | en:developpement:productions:sidus [2020/08/17 13:52] (Version actuelle) – supprimée equemene | ||
|---|---|---|---|
| Ligne 1: | Ligne 1: | ||
| - | ====== SIDUS : Single Instance Distributing Universal System ====== | ||
| - | |||
| - | <note important> | ||
| - | |||
| - | {{ : | ||
| - | |||
| - | <note warning> | ||
| - | |||
| - | <note tip> | ||
| - | |||
| - | **SIDUS** is the contraction of //Single Instance Universal Distributing System// and purposes to oversimplify the administration of large amount of machines. | ||
| - | |||
| - | Its [[http:// | ||
| - | |||
| - | **SIDUS** has two fondamental properties : | ||
| - | * **unicity of configuration** : two machines starting SIDUS run exactly the same Operating System. | ||
| - | * **use of local resources** : processors and RAM used are local host one. | ||
| - | |||
| - | **SIDUS** is not : | ||
| - | * **neither LTSP** for //Linux Terminal Server Project//: LTSP offers simplified management of thin clients by providing a X11 or RDP access to a server: and it supports the entire processing load. Conversely, SIDUS operates entirely (or at the discretion of the user) any machine that hangs there. Only the storage operating system is offset to third machines. | ||
| - | * **neither FAI** for //Fully Automatic Installation//: | ||
| - | * **neither a network LiveCD**: LiveCD starts a minimalist system, necessarily fixed. It is always possible to create your own LiveCD but it is a tricky operation. With SIDUS, it is possible to install on the fly for all users new software. | ||
| - | |||
| - | ===== Five Ws & Two Hs ===== | ||
| - | |||
| - | ===== Why ? ===== | ||
| - | |||
| - | Time of administration system of computer equipment (compute node, post or personal workstation, | ||
| - | |||
| - | This is the challenge faced by SIDUS (for Single Instance Universal Distributing System), developed at the Centre Blaise Pascal originally mainly to simplify the task of the system administrator only deal with the management of hundreds of machines of all kinds for all destinations (technical platforms multi-node, multi-core, GPU, etc.). | ||
| - | |||
| - | ===== What ? ===== | ||
| - | |||
| - | One approach to network booting and provides a perfectly generic system for all machines. | ||
| - | |||
| - | ===== When ? ===== | ||
| - | |||
| - | The first version of SIDUS of February 2010, it was originally on Debian Etch. It followed all the developments of the Debian distribution, | ||
| - | |||
| - | Early 2013, SIDUS serves nearly 90 permanent cluster nodes to CBP (up to 120 simultaneous based loans), GPGPU servers, workstations and multi-core positions //COMOD// (for //Compute On My Own Device//). | ||
| - | |||
| - | ===== For Who ? ===== | ||
| - | |||
| - | SIDUS is for all those who want to simplify the task and that: | ||
| - | * Have groups of machines having all the same destination: | ||
| - | * Nodes computing cluster | ||
| - | * Machines free service room | ||
| - | * Virtual machines in training | ||
| - | * Want to analyze equipment without ever touching the system | ||
| - | |||
| - | ===== Where ? ===== | ||
| - | |||
| - | The Centre Blaise Pascal, hotel projects ENS-Lyon in the field of computing and scientific computing, SIDUS uses for all its amenities including uniformity must be preserved as much as possible: a simple reboot should be sufficient to replace the system in its original condition. | ||
| - | |||
| - | The Scientific Centre of Numerical Modeling (PSMN) computing center ENS-Lyon, now uses SIDUS over a hundred knots and prepares its generalization for the establishment of Meso@Equip (nearly 200 additional nodes). On september 2013, the number of nodes working under SIDUS exceeds 320. | ||
| - | |||
| - | The chemistry laboratory uses " | ||
| - | |||
| - | Members of the laboratory at the institute, and the LBMC and LJC are interested in " | ||
| - | |||
| - | University Joseph Fourier, through its thematic schools on scientific computing, used since 2011 for infrastructure SIDUS work of auditors. | ||
| - | |||
| - | ===== How much ? ===== | ||
| - | |||
| - | 8 Neoware thin clients inflated CPU and memory in early 2010 and diverted from their original purpose, we are approaching the 120 machines to CBP using this system. | ||
| - | |||
| - | Some researchers use chemical laboratory via SIDUS COMOD: availability by providing the ability to deploy a full working machine on the job in seconds. | ||
| - | |||
| - | When thematic Les Houches School on scientific computing, SIDUS was served nearly fifty machines simultaneously. | ||
| - | |||
| - | As for the price of the software (question asked last Scipy 2013), all the components that it uses is Open Source, SIDUS is also, as well as all related documentation! SIDUS is under CeCILL. | ||
| - | |||
| - | If you use SIDUS, notify the [[emmanuel.quemener @ ens-lyon.fr | author]] and make the promotion! | ||
| - | |||
| - | If you want to deploy SIDUS on your facilities and get help, you can contact the [[emmanuel.quemener@ens-lyon.fr| author]]. | ||
| - | |||
| - | ===== How ? ===== | ||
| - | |||
| - | SIDUS is based on a simple majority and proven components available on most GNU / Linux distributions. | ||
| - | |||
| - | We will now describe how to install our own SIDUS! | ||
| - | |||
| - | ====== Installation ====== | ||
| - | |||
| - | ===== Preparing the System ===== | ||
| - | |||
| - | We need to prepare a bit to accommodate our SIDUS. For this, we hand over several services to deploy our clients: DHCP, TFTP, NFS servers. You maintain a good relationship with your IT department or you are free enough to unconstrained access to LDAP and DNS servers defined: | ||
| - | |||
| - | * DHCP service provides our client an IP address but diffuse and additional information: | ||
| - | * TFTP service when the scene. It offers everything needed by TFTP for booting the system: the binary '' | ||
| - | * NFS server then calls in the loop: it will provide the root system by its protocol (so nfsroot). It is in this root, for example ''/ | ||
| - | |||
| - | Our configurations we use respectively **isc-dhcp-server**, | ||
| - | |||
| - | Look in more detail the configuration of the different services we have include: | ||
| - | |||
| - | <note important> | ||
| - | For the DHCP server in the configuration file ''/ | ||
| - | next-server 10.13.20.13; | ||
| - | filename " | ||
| - | allow booting;</ | ||
| - | |||
| - | For the TFTP server in the ''/ | ||
| - | ./ | ||
| - | ./ | ||
| - | ./ | ||
| - | ./ | ||
| - | </ | ||
| - | |||
| - | In the '' | ||
| - | * the kernel '' | ||
| - | * the bootstrap system '' | ||
| - | * the NFSroot server '' | ||
| - | |||
| - | < | ||
| - | DEFAULT tmpfs | ||
| - | |||
| - | LABEL tmpfs | ||
| - | KERNEL vmlinuz-Sidus | ||
| - | APPEND console=tty1 root=/ | ||
| - | |||
| - | LABEL iscsi | ||
| - | KERNEL vmlinuz-Sidus | ||
| - | APPEND console=tty1 root=/ | ||
| - | </ | ||
| - | |||
| - | For the NFS server, the configuration takes a line in the file / etc / exports. Here we open a read-only access to the machines is between the IP 10.13.20.128 and 10.13.20.254:< | ||
| - | / | ||
| - | </ | ||
| - | |||
| - | Once these three DHCP, TFTP and NFS services configured we can install a complete SIDUS. | ||
| - | |||
| - | Be careful though! It will also require a root containing the accounts of users (NFS version 4) and a process for their identification/ | ||
| - | |||
| - | ===== Deboostrap ===== | ||
| - | |||
| - | Debootstrap For installation of a system as a root. It requires several parameters such as the installation root, the hardware architecture, | ||
| - | |||
| - | Warning: there begins the " | ||
| - | |||
| - | Debootstrap also accepts as input a list of archives (you know Debian is very fussy about licensing separating archives main, contrib and non-free), a list of packages to be included and a list of packages to exclude. We would have been delighted to here, specify the complete list of packages to include or exclude, but unfortunately, | ||
| - | |||
| - | We have for convenience defined environment variables corresponding to the root of our system '' | ||
| - | |||
| - | < | ||
| - | export SIDUS=/ | ||
| - | alias sidus=" | ||
| - | </ | ||
| - | |||
| - | Here are examples of '' | ||
| - | < | ||
| - | export MyInclude=" | ||
| - | |||
| - | export MyExclude=" | ||
| - | </ | ||
| - | |||
| - | By default, the next command install un Debian Wheezy distribution in **/ | ||
| - | < | ||
| - | time debootstrap --arch amd64 --components=' | ||
| - | </ | ||
| - | |||
| - | Following this command, we need to take some precautions: | ||
| - | * normally, if the Debian package is a service, it starts after installation. We must therefore inhibit the launch of this service by defining a hook:< | ||
| - | printf '# | ||
| - | chmod +x ${SIDUS}/ | ||
| - | </ | ||
| - | * packages require access to the list of processes, systems, peripherals, | ||
| - | sidus mount -t proc none /proc | ||
| - | sidus mount -t sysfs sys /sys | ||
| - | mount --bind /run/shm ${SIDUS}/ | ||
| - | mount --bind /dev/pts ${SIDUS}/ | ||
| - | </ | ||
| - | |||
| - | ===== Specific packages installation ===== | ||
| - | |||
| - | In order to simplify the installation of packets belonging to the same family, has created many Debian meta packages, prefixed with " | ||
| - | time sidus apt-get install --install-suggests -f -m -y --force-yes science-*</ | ||
| - | |||
| - | During installation, | ||
| - | |||
| - | In the best case scenario, it takes 45 minutes to a full tree 32GB. | ||
| - | |||
| - | ===== Purge ===== | ||
| - | |||
| - | Unfortunately, | ||
| - | time sidus apt-get purge -y -f --force-yes matlab-*</ | ||
| - | |||
| - | ===== Environnement local ===== | ||
| - | |||
| - | The local environment is significant and, by default, nothing is set. The default is the U.S., we parameterize: | ||
| - | * local environment: | ||
| - | * timezone : '' | ||
| - | * keyboard : '' | ||
| - | |||
| - | ===== LDAP & NFS third party servers ===== | ||
| - | |||
| - | LDAP and NFS services require a small configuration. Import prefined files is better. | ||
| - | |||
| - | In our case, we chose to put them all on one Web server **http:// | ||
| - | wget -O ${SIDUS}/ | ||
| - | </ | ||
| - | |||
| - | So, here are the files to download: | ||
| - | |||
| - | * The LDAP authentication: | ||
| - | * '' | ||
| - | * '' | ||
| - | * '' | ||
| - | * '' | ||
| - | * The NFS folders for users : | ||
| - | * for NFSv4 : | ||
| - | * '' | ||
| - | * '' | ||
| - | * '' | ||
| - | * for NFSv3 : | ||
| - | * '' | ||
| - | |||
| - | ===== Startup sequence ===== | ||
| - | |||
| - | How SIDUS sharing without duplicating? | ||
| - | |||
| - | Everything is in a single " | ||
| - | - creating temporary files ''/ | ||
| - | - moving NFSroot from initial mount point to ''/ | ||
| - | - mounting a remote share, a local or remote partition, or TMPFS volume to another mount point ''/ | ||
| - | - superposition of the two folders ''/ | ||
| - | - moving ''/ | ||
| - | |||
| - | This script, '' | ||
| - | |||
| - | <note important> | ||
| - | The original script was inspired by the project **rootaufs** from Nicholas A. Schembri (http:// | ||
| - | < | ||
| - | wget -O ${SIDUS}/ | ||
| - | </ | ||
| - | |||
| - | We have not finished yet to have a functional system we create a special '' | ||
| - | * '' | ||
| - | * '' | ||
| - | |||
| - | < | ||
| - | |||
| - | <note important> | ||
| - | Then just copy the kernel and boot loader in the definition : | ||
| - | < | ||
| - | cp ${SIDUS}/ | ||
| - | cp ${SIDUS}/ | ||
| - | </ | ||
| - | |||
| - | Initially, we explored the possibility of offering a second NFS share as read/write persistence associated with a client reboots changes. This version, though functional, demanded the opening of an atomic NFS from each customer with what we imagine as a burden on their server! | ||
| - | |||
| - | We have preferred another approach persistence in the form of a network of iSCSI disk, which we saw in our definition file ''/ | ||
| - | |||
| - | For reasons of simplicity, the volume has offered the client IP and we provide only the server iSCSI volume. The login and password to access the default password is in the file '' | ||
| - | |||
| - | ===== Some tricks ===== | ||
| - | |||
| - | * for configuring the DHCP name (hostname), remove ''/ | ||
| - | * set the ''/ | ||
| - | * in ''/ | ||
| - | * modify GDM3 to start necessary after NSCD and DBUS daemons | ||
| - | * set ''/ | ||
| - | * set ''/ | ||
| - | |||
| - | ==== For virtual systems based on VirtualBox ==== | ||
| - | |||
| - | * Install VBoxLinuxAdditions.run in SIDUS system | ||
| - | |||
| - | ==== For systems with InfiniBand card ==== | ||
| - | |||
| - | * Force loading modules in ''/ | ||
| - | * Insert in ''/ | ||
| - | |||
| - | ==== For systems with Nvidia graphics boards ==== | ||
| - | |||
| - | For the majority of Nvidia cards, packages available in Debian Wheezy allow complete installation owners drivers, libraries OpenGL, OpenCL and CUDA. Be careful though if you want to use the ICD (Installable Client Loader) OpenCL AMD processors simultaneously operate your AND your graphics card: we had to install yourselves all the infrastructure Cuda and OpenCL "from scratch" | ||
| - | |||
| - | ==== For systems with AMD/ATI graphics boards ==== | ||
| - | |||
| - | For the majority of ATI cards, packages available in Debian Wheezy allow a full install proprietary drivers, libraries OpenGL, OpenCL and CUDA. | ||
| - | |||
| - | ===== Removing binding " | ||
| - | |||
| - | In order to properly install SIDUS, we were forced to bind host and chroot closely. | ||
| - | |||
| - | Here are the commands to set the system as original one: | ||
| - | < | ||
| - | umount ${SIDUS}/ | ||
| - | umount ${SIDUS}/ | ||
| - | sidus umount / | ||
| - | sidus umount /proc | ||
| - | sidus umount /sys | ||
| - | </ | ||
| - | |||
| - | ===== Deleting temporary folders ===== | ||
| - | |||
| - | < | ||
| - | rm -r ${SIDUS}/ | ||
| - | </ | ||
| - | |||
| - | ===== Purge processes related to SIDUS ===== | ||
| - | |||
| - | < | ||
| - | lsof | grep ${SIDUS} | ||
| - | </ | ||
| - | |||
| - | ====== Administration ====== | ||
| - | |||
| - | The administration of a SIDUS instance is administered like any system chroot: it however are precautions to take, all the administrative functions that do not require the same constraints from each other. | ||
| - | |||
| - | ===== Update Debian ===== | ||
| - | |||
| - | This is certainly the most common function. The problem of up-to-date is twofold: | ||
| - | * Packages can replace services starting with ''/ | ||
| - | * Packages require the presence of a ''/ | ||
| - | |||
| - | To achieve up-to-date a SIDUS instance installed in '' | ||
| - | |||
| - | * Before update commands:< | ||
| - | mount --bind /dev/shm ${SIDUS}/ | ||
| - | mount --bind /dev/pts ${SIDUS}/ | ||
| - | chroot ${SIDUS} mount -t proc none /proc | ||
| - | </ | ||
| - | * Update commands :< | ||
| - | chroot ${SIDUS} aptitude update | ||
| - | chroot ${SIDUS} aptitude -y full-upgrade | ||
| - | chroot ${SIDUS} aptitude clean</ | ||
| - | * After update commands :< | ||
| - | umount -l /proc | ||
| - | umount -l ${SIDUS}/ | ||
| - | umount -l ${SIDUS}/ | ||
| - | lsof | grep ${SIDUS} | ||
| - | </ | ||
| - | |||
| - | The last line of the downstream control is a precaution to prevent the process started in the instance running on the host. | ||
| - | |||
| - | |||
| - | ====== Demonstration for Scipy 2013 ====== | ||
| - | |||
| - | Next images[[http:// | ||
| - | |||
| - | These two images require both the open source virtualization tool [[http:// | ||
| - | |||
| - | The SIDUS instance is integrated into the virtual machine **sidus4scipy** | ||
| - | |||
| - | The virtual machine **sidus4client** | ||
| - | |||
| - | Some elements: | ||
| - | * Space required to install the complete image is 8GB | ||
| - | * To import: | ||
| - | * Ask resetting the Mac address (otherwise, all persons on the same network have the same IP address: communication difficulties are to be expected :) | ||
| - | * Before starting the virtual machine: | ||
| - | * Open settings | ||
| - | * Go into the network settings and check the network interface is eth0 ** ** Linux ** ** en0 MacOS | ||
| - | * Go into the settings and create a shared folder myhost ** ** | ||
| - | * It is preferable to have at least 3 GB on the machine creating two virtual machines | ||
| - | * During startup, the U.S. QWERTY keyboards | ||
| - | * Usernames and passwords for the root and generic accounts created 4 | ||
| - | * **root** & **Sidus2013** | ||
| - | * **alpha** & **Alpha13** | ||
| - | * **beta** & **Beta13** | ||
| - | * **gamma** & **Gamma13** | ||
| - | * **delta** & **Delta13** | ||
| - | * (see all other Greek letters in a dictionary!) | ||
| - | * **omega** & **Omega13** | ||
| - | |||
| - | <note warning> | ||
| - | * The default keyboard is an American Qwerty | ||
| - | * The default language is English | ||
| - | * The time zone of the Austin | ||
| - | It is very easy to iron in a " | ||
| - | * For SIDUS server | ||
| - | *'' | ||
| - | *'' | ||
| - | * For customer SIDUS | ||
| - | *'' | ||
| - | *'' | ||
| - | </ | ||
| - | |||
| - | ===== Starting on third party machine ===== | ||
| - | |||
| - | It is also possible to start the instance on another machine by selecting the configuration of the virtual machine network that you intend to use (usually '' | ||
| - | * Either by directly connecting the machine to another via the Ethernet port, a necessary condition is | ||
| - | * RJ45 port supports autoconfiguration in cross harbor | ||
| - | * Either by connecting the host computer and the client device on an external switch | ||
| - | |||
| - | In both cases, // | ||
| - | |||
| - | --- // | ||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||
| - | |||