Différences

Ci-dessous, les différences entre deux révisions de la page.

Lien vers cette vue comparative

Les deux révisions précédentes Révision précédente
Prochaine révision
Révision précédente
Prochaine révision Les deux révisions suivantes
en:developpement:productions:sidus [2013/09/28 22:45]
equemene [LDAP & NFS third party servers]
en:developpement:productions:sidus [2014/11/04 17:22]
equemene [Startup sequence]
Ligne 1: Ligne 1:
 ====== SIDUS : Single Instance Distributing Universal System ====== ====== SIDUS : Single Instance Distributing Universal System ======
  
-{{ :developpement:​productions:​sidus.png?​200 ​|}}+<note important>​For proper English, please refer to this Linux Journal [[http://www.linuxjournal.com/​content/​november-2013-issue-linux-journal-system-administration|article]] published in November 2013. It was written by Marianne Corvellec based on Emmanuel Quemener'​s articles.</​note>​
  
-<note important>​Work in progress</​note>​+{{ :​developpement:​productions:​sidus.png?​200 |}}
  
-<note warning>​All items in this documentation,​ code snippets, etc. are part of the license [[http://​www.cecill.info/​index.fr.html|CeCILL]]. It is therefore necessary to respect the four freedoms of free software to operate ​RESIDUE ​in its own infrastructure. If you intend to use in your RESIDUE ​infrastructure,​ let her know [[emmanuel.quemener @ ens-lyon.fr | author]], Centre Blaise Pascal or quote it!</​note>​+<note warning>​All items in this documentation,​ code snippets, etc. are part of the license [[http://​www.cecill.info/​index.fr.html|CeCILL]]. It is therefore necessary to respect the four freedoms of free software to operate ​SIDUS in its own infrastructure. If you intend to use in your SIDUS infrastructure,​ let her know [[emmanuel.quemener @ ens-lyon.fr | author]], Centre Blaise Pascal or quote it!</​note>​
  
 <note tip>​First [[https://​conference.scipy.org/​scipy2013/​presentation_detail.php?​id=199|presentation]] with [[http://​www.youtube.com/​watch?​v=J5myH0y_bks|video]] of **SIDUS** on [[http://​conference.scipy.org/​scipy2013|Scipy 2013]]</​note>​ <note tip>​First [[https://​conference.scipy.org/​scipy2013/​presentation_detail.php?​id=199|presentation]] with [[http://​www.youtube.com/​watch?​v=J5myH0y_bks|video]] of **SIDUS** on [[http://​conference.scipy.org/​scipy2013|Scipy 2013]]</​note>​
Ligne 17: Ligne 17:
   * **use of local resources** : processors and RAM used are local host one.   * **use of local resources** : processors and RAM used are local host one.
  
-**SIDUS** ​n'est donc +**SIDUS** ​is not 
-  * **neither LTSP** for //Linux Terminal Server Project//: LTSP provides ​simplified management of thin clients by providing a X11 or RDP access to a server: and it supports the entire processing load. Conversely, SIDUS operates entirely (or at the discretion of the user) any machine that hangs there. Only the storage operating system is offset to third machines. +  * **neither LTSP** for //Linux Terminal Server Project//: LTSP offers ​simplified management of thin clients by providing a X11 or RDP access to a server: and it supports the entire processing load. Conversely, SIDUS operates entirely (or at the discretion of the user) any machine that hangs there. Only the storage operating system is offset to third machines. 
-  * **neither FAI** for //Fully Automatic Installation//:​ FAI or Kickstart offer a complete easy installation to minimize or eliminate any action by the administrator. Conversely, SIDUS offers a unique system in a tree incorporating both the base system and all installed applications manually. +  * **neither FAI** for //Fully Automatic Installation//:​ FAI and Kickstart offer a complete easy installation to minimize or eliminate any action by the administrator. Conversely, SIDUS offers a unique system in a tree incorporating both the base system and all installed applications manually. 
-  * **neither a network LiveCD**: LiveCD starts a minimalist system, necessarily fixed. It is always possible to create your own LiveCD but it is a fairly expensive ​operation. With SIDUS, it is possible to install the fly on all its customers instantly ​software ​to reconfigure.+  * **neither a network LiveCD**: LiveCD starts a minimalist system, necessarily fixed. It is always possible to create your own LiveCD but it is a tricky ​operation. With SIDUS, it is possible to install ​on the fly for all users new software.
  
 ===== Five Ws & Two Hs ===== ===== Five Ws & Two Hs =====
Ligne 28: Ligne 28:
 Time of administration system of computer equipment (compute node, post or personal workstation,​ virtual machine testing) increases with the number and diversity. Thus, all these materials essentially share the same files, but each on its own disk. How to limit the installation time and administrative machinery while retaining the flexibility related to their destination?​ Time of administration system of computer equipment (compute node, post or personal workstation,​ virtual machine testing) increases with the number and diversity. Thus, all these materials essentially share the same files, but each on its own disk. How to limit the installation time and administrative machinery while retaining the flexibility related to their destination?​
  
-This is the challenge faced by RESIDUE ​(for Single Instance Universal Distributing System), developed at the Centre Blaise Pascal originally mainly to simplify the task of the system administrator only deal with the management of hundreds of machines of all kinds for all destinations (technical platforms multi-node, multi-core, GPU, etc.).+This is the challenge faced by SIDUS (for Single Instance Universal Distributing System), developed at the Centre Blaise Pascal originally mainly to simplify the task of the system administrator only deal with the management of hundreds of machines of all kinds for all destinations (technical platforms multi-node, multi-core, GPU, etc.).
  
 ===== What ? ===== ===== What ? =====
Ligne 77: Ligne 77:
 ===== How ? ===== ===== How ? =====
  
-RESIDUE ​is based on a simple majority and proven components available on most GNU / Linux distributions.+SIDUS is based on a simple majority and proven components available on most GNU / Linux distributions.
  
 We will now describe how to install our own SIDUS! We will now describe how to install our own SIDUS!
Ligne 95: Ligne 95:
 Look in more detail the configuration of the different services we have include: Look in more detail the configuration of the different services we have include:
  
 +<note important>​In [[http://​www.linuxjournal.com/​content/​november-2013-issue-linux-journal-system-administration|Linux Journal]] article, next-server is defined as ''​172.16.20.13'',​ it's better to put it inside LAN: 10.13.20.13 is a better choice.</​note>​
 For the DHCP server in the configuration file ''/​etc/​dhcp/​dhcpd.conf''​ :<​code>​ For the DHCP server in the configuration file ''/​etc/​dhcp/​dhcpd.conf''​ :<​code>​
 next-server 10.13.20.13;​ next-server 10.13.20.13;​
Ligne 145: Ligne 146:
 export SIDUS=/​srv/​nfsroot/​sidus export SIDUS=/​srv/​nfsroot/​sidus
 alias sidus="​DEBIAN_FRONTEND=noninteractive chroot ${SIDUS} $@" alias sidus="​DEBIAN_FRONTEND=noninteractive chroot ${SIDUS} $@"
 +</​code>​
 +
 +Here are examples of ''​MyInclude''​ and ''​MyExclude''​ we use
 +<​code>​
 +export MyInclude="​adduser,​apt,​apt-utils,​aptitude,​aptitude-common,​aspell,​aspell-en,​aufs-tools,​bsdmainutils,​btrfs-tools,​busybox,​ca-certificates,​clusterssh,​console-setup,​cpio,​cron,​cups-pdf,​debian-archive-keyring,​dmidecode,​dselect,​emacs,​environment-modules,​ethtool,​firmware-bnx2,​firmware-linux,​firmware-linux-nonfree,​gnupg,​gpgv,​groff-base,​htop,​hwinfo,​hwloc,​iftop,​ifupdown,​info,​initramfs-tools,​install-info,​iotop,​iperf,​ipmitool,​iproute,​iptables,​iputils-ping,​isc-dhcp-client,​isc-dhcp-common,​kmod,​ldap-utils,​less,​libapt-inst1.5,​libapt-pkg4.12,​libboost-iostreams1.49.0,​libcwidget3,​libept1.4.12,​libgcrypt11,​libgdbm3,​libgnutls26,​libgpg-error0,​libidn11,​libkmod2,​libncursesw5,​libnet-ldap-perl,​libnewt0.52,​libnfnetlink0,​libnss-ldap,​libp11-kit0,​libpam-ldap,​libpipeline1,​libpopt0,​libprocps0,​libreadline6,​libsigc++-2.0-0c2a,​libsqlite3-0,​libssl1.0.0,​libstdc++6,​libtasn1-3,​libudev0,​libusb-0.1-4,​libxapian22,​linux-headers-3.2.0-4-amd64,​linux-image-3.2.0-4-amd64,​locales,​logrotate,​lsof,​man-db,​manpages,​mbw,​mtr,​mutt,​nano,​net-tools,​netbase,​netcat-traditional,​nfs-common,​nscd,​ntpdate,​open-iscsi,​openssh-server,​pciutils,​procps,​python-ldap,​readline-common,​rsyslog,​screen,​scsitools,​sdparm,​ssh,​ssmtp,​sudo,​tasksel,​tasksel-data,​tmux,​traceroute,​tshark,​udev,​usbutils,​vim,​wget,​whiptail,​xinit,​python-html2text"​
 +
 +export MyExclude="​nano,​exim,​mysql-server,​mysql-server-5.5,​mysql-server-core-5.5,​network-manager,​apache2,​apache2-mpm-worker,​apache2-utils,​apache2.2-bin,​apache2.2-common,​libapache2-mod-dnssd,​libapache2-mod-php5,​r-cran-fecofin,​libmpich1.0gf,​gerris,​gspiceui,​qucs,​ktimetrace,​kseg,​ghdl,​earth3d,​libopenigtlink1,​qtdmm,​scilab-overload,​gmsh,​klogic,​g++-doc,​openturns-wrapper,​xorsa,​r-cran-rpvm,​labplot,​zygrib,​libteem1,​magnus,​libcomplearn-dev,​libtorque2,​torque-common,​torque-server,​gridengine-client,​gridengine-exec,​gridengine-master,​gridengine-qmon,​gnuplot,​gnuplot-nox,​rtai,​rtai-doc,​libhdf5-dev,​libhdf5-1.8,​libgd2-xpm"​
 </​code>​ </​code>​
  
Ligne 175: Ligne 183:
 ===== Purge ===== ===== Purge =====
  
-Unfortunately,​ this installation bulimia is not without effect. Packages still install some "​evil"​ and a purge of a few, including an installer Matlab, we ruffles hair!<​code>​+Unfortunately,​ this installation bulimia is not without effect. Packages still install some "​evil"​ and a purge of a few, including an installer Matlab!<​code>​
 time sidus apt-get purge -y -f --force-yes matlab-*</​code>​ time sidus apt-get purge -y -f --force-yes matlab-*</​code>​
  
Ligne 221: Ligne 229:
 This script, ''​rootaufs'',​ in located in ''​${SIDUS}/​etc/​initramfs-tools/​scripts/​init-bottom''​ This script, ''​rootaufs'',​ in located in ''​${SIDUS}/​etc/​initramfs-tools/​scripts/​init-bottom''​
  
-The original script was inspired by the project **rootaufs** from Nicholas A. Schembri (http://​code.google.com/​p/​rootaufs/​). He was deeply modified to adapt our infrastructure:​ a version is available on http://​www.cbp.ens-lyon.fr/​sidus/​rootaufs :<​code>​ +<note important>​In [[http://​www.linuxjournal.com/​content/​november-2013-issue-linux-journal-system-administration|Linux Journal]] article, ''​rootaufs''​ as destination file is missing.</​note>​ 
-wget -O ${SIDUS}/​etc/​initramfs-tools/​scripts/​init-bottom http://​www.cbp.ens-lyon.fr/​sidus/​rootaufs+The original script was inspired by the project **rootaufs** from Nicholas A. Schembri (http://​code.google.com/​p/​rootaufs/​). He was deeply modified to adapt our infrastructure:​ a version is available on http://​www.cbp.ens-lyon.fr/​sidus/​rootaufs : 
 +<​code>​ 
 +wget -O ${SIDUS}/​etc/​initramfs-tools/​scripts/​init-bottom/​rootaufs ​http://​www.cbp.ens-lyon.fr/​sidus/​rootaufs
 </​code>​ </​code>​
  
Ligne 231: Ligne 241:
 <​code>​sidus update-initramfs -k all -u</​code>​ <​code>​sidus update-initramfs -k all -u</​code>​
  
-Then just copy the kernel and boot loader in the definition :<​code>​ +<note important>​In [[http://​www.linuxjournal.com/​content/​november-2013-issue-linux-journal-system-administration|Linux Journal]] article, ''​vmlinux-Sidus''​ is written, not ''​vmlinuz-Sidus''​ as TFTP kernel file.</​note>​ 
-cp ${SIDUS}/​vmlinuz /srv/tftp/vmlinux-Sidus +Then just copy the kernel and boot loader in the definition : 
-cp ${SIDUS}/​srv/​nfsroot/​boot/​initrd /​srv/​tftp/​initrd-Sidus+<​code>​ 
 +cp ${SIDUS}/boot/vmlinuz-3.2.0-4-amd64 ​/srv/tftp/vmlinuz-Sidus 
 +cp ${SIDUS}/​boot/​initrd.img-3.2.0-4-amd64 ​/​srv/​tftp/​initrd.img-Sidus
 </​code>​ </​code>​
  
Ligne 262: Ligne 274:
 ==== For systems with Nvidia graphics boards ==== ==== For systems with Nvidia graphics boards ====
  
-For the majority of Nvidia cards, packages available in Debian Wheezy allow complete installation owners drivers, libraries OpenGL, OpenCL and CUDA. Be careful though if you want to use the ICD (Installable Client Loader) OpenCL AMD processors simultaneously operate your AND your graphics card: we had to install all the infrastructure ​pilot Cuda and OpenCL "from scratch" ​afford.+For the majority of Nvidia cards, packages available in Debian Wheezy allow complete installation owners drivers, libraries OpenGL, OpenCL and CUDA. Be careful though if you want to use the ICD (Installable Client Loader) OpenCL AMD processors simultaneously operate your AND your graphics card: we had to install ​yourselves ​all the infrastructure Cuda and OpenCL "from scratch"​.
  
 ==== For systems with AMD/ATI graphics boards ==== ==== For systems with AMD/ATI graphics boards ====
Ligne 270: Ligne 282:
 ===== Removing binding "​system"​ ===== ===== Removing binding "​system"​ =====
  
-In order to properly install ​RESIDUE, we were forced to bind host and chroot closely.+In order to properly install ​SIDUS, we were forced to bind host and chroot closely.
  
 Here are the commands to set the system as original one: Here are the commands to set the system as original one:
Ligne 326: Ligne 338:
 ====== Demonstration for Scipy 2013 ====== ====== Demonstration for Scipy 2013 ======
  
-Les images ​suivantes ​[[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4scipy2013.ova|Sidus4Scipy2013]] ​et [[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4client.ova|Sidus4Client]] ​permettent de juger de l'​efficacité de Sidus à offrir un environnement complet sur un poste de travail unique.+Next images[[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4scipy2013.ova|Sidus4Scipy2013]] ​and [[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4client.ova|Sidus4Client]] ​can judge the effectiveness of Sidus to provide a complete environment on a single workstation.
  
-Ces deux images ​nécessitent toutes les deux l'​outil de virtualisation Open Source ​[[http://​www.virtualbox.org|VirtualBox]] ​disponibles sur les plates-formes ​Linux, MacOS X, Windows ​et Solaris. ​+These two images ​require both the open source virtualization tool [[http://​www.virtualbox.org|VirtualBox]] ​available on Linux platforms, MacOS X, Windows ​and Solaris.
  
-L'​instance ​SIDUS est intégrée dans la machine ​virtuelle ​**sidus4scipy** ​issue de l'​importation de la machine [[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4scipy2013.ova|Sidus4Scipy]]. ​Elle offre un bureau ​Gnome complet avec des outils ​Python ​présentés lors de Scipy 2013+The SIDUS instance is integrated into the virtual ​machine **sidus4scipy** ​ after import of machine [[http://​www.cbp.ens-lyon.fr/​sidus/​sidus4scipy2013.ova|Sidus4Scipy]]. ​It offers a full Gnome desktop with Python ​tools presented at Scipy 2013.
  
-La machine ​virtuelle ​**sidus4client** ​intègre la machine ​virtuelle minimale permettant de démarrer un client ​utilisant l'​instance ​SIDUS. ​Après quelques secondesun bureau ​Gnome s'​ouvre.+The virtual ​machine **sidus4client** ​ ​integrates the minimum virtual ​machine ​to start a client ​using SIDUS instanceAfter a few secondsGnome desktop opens.
  
-Quelques éléments ​+Some elements
-  * l'​espace nécessaire pour installer l'image complète est de 8 Go +  * Space required to install the complete ​image is 8GB 
-  * à l'​importation ​+  * To import
-    * demander la réinitialisation de l'​adresse ​Mac (dans le cas contrairetoutes les personnes sur le même réseau local auront la même adresse ​IP : des difficultés de communication ​sont à prévoir ​:) +    * Ask resetting the Mac address ​(otherwiseall persons on the same network have the same IP address: communication ​difficulties are to be expected ​:) 
-  * avant le démarrage de la machine ​virtuelle ​+  * Before starting the virtual ​machine: 
-    * ouvrir les paramétrages +    * Open settings 
-    * aller dans le paramétrage réseau et vérifier l'interface ​réseau : c'​est ​**eth0** sous Linux**en0** sous MacOS +    * Go into the network settings and check the network ​interface ​is eth0 ** ** Linux ** ** en0 MacOS 
-    * aller dans le paramétrage de partage et créer un dossier ​**MyHost** +    * Go into the settings and create a shared folder myhost ​** ** 
-  * il est préférable de disposer d'au moins Go sur la machine ​créant ces deux machines ​virtuelles +  * It is preferable to have at least GB on the machine ​creating two virtual ​machines 
-  * lors du démarrageles claviers sont en QWERTY ​américains +  * During startupthe U.S. QWERTY ​keyboards 
-  * les identifiants et mots de passe du root et de comptes génériques créés +  * Usernames and passwords for the root and generic accounts created ​
-    * **root** ​et **Sidus2013** +    * **root** ​**Sidus2013** 
-    * **alpha** ​et **Alpha13** +    * **alpha** ​**Alpha13** 
-    * **beta** ​et **Beta13** +    * **beta** ​**Beta13** 
-    * **gamma** ​et **Gamma13** +    * **gamma** ​**Gamma13** 
-    * **delta** ​et **Delta13** +    * **delta** ​**Delta13** 
-    * (voir pour toutes les autres lettres grecques dans un dico !) +    * (see all other Greek letters in a dictionary!) 
-    * **omega** ​et **Omega13**+    * **omega** ​**Omega13**
  
-<note warning>Attention ​Dans un souci d'​internationalisation ​(Scipy 2013) +<note warning>CautionIn an effort to internationalize ​(Scipy 2013) 
-  * le clavier par défaut est un américain ​Qwerty +  * The default keyboard is an American ​Qwerty 
-  * la langue par défaut est l'​anglais +  * The default language is English 
-  * le fuseau horaire celui de Austin +  * The time zone of the Austin 
-Il est très facile de repasser dans un mode "français" ​pour le clavier et le fuseau ​via les commandes +It is very easy to iron in a "French" ​mode for the keyboard and time via commands 
-  * Pour le serveur ​SIDUS +  * For SIDUS server 
-    * ''​dpkg-reconfigure keyboard-configuration''​ +    *''​dpkg-reconfigure keyboard-configuration''​ 
-    * ''​dpkg-reconfigure tzdata''​ +    *''​dpkg-reconfigure tzdata''​ 
-  * Pour le client ​SIDUS +  * For customer ​SIDUS 
-    * ''​sidus dpkg-reconfigure keyboard-configuration''​ +    *''​sidus dpkg-reconfigure keyboard-configuration''​ 
-    * ''​sidus dpkg-reconfigure tzdata''​+    *''​sidus dpkg-reconfigure tzdata''​
 </​note>​ </​note>​
  
-===== Démarrage sur une machine ​tierce ​=====+===== Starting on third party machine =====
  
-Il est aussi possible ​de démarrer l'instance ​sur une autre machine ​en sélectionnant dans la configuration ​de la machine ​virtuelle l'​interface réseau que vous comptez utiliser ​(généralement ​''​eth0''​),​  +It is also possible ​to start the instance ​on another ​machine ​by selecting the configuration ​of the virtual ​machine ​network that you intend to use (usually ​''​eth0''​) ​interface
-  * soit en connectant directement la machine ​à une autre via le port Ethernetune condition ​est nécessaire +  * Either by directly connecting the machine ​to another ​via the Ethernet ​port, a necessary ​condition ​is 
-    * un port RJ45 supportant l'autoconfiguration ​en port croisé +    * RJ45 port supports ​autoconfiguration ​in cross harbor 
-  * soit en connectant la machine hôte et la machine cliente sur un commutateur externe+  * Either by connecting the host computer and the client device on an external switch
  
-Dans les deux casle //​forwarding// ​doit être activé ​: ''/​proc/​sys/​net/​ipv4/​ip_forward'' ​mis à ''​1''​.+In both cases, //​forwarding// ​must be activated ​: ''/​proc/​sys/​net/​ipv4/​ip_forward'' ​set to ''​1''​.
  
- --- //​[[emmanuel.quemener@ens-lyon.fr|Emmanuel Quemener]] 2013/07/02 18:08//+ --- //​[[emmanuel.quemener@ens-lyon.fr|Emmanuel Quemener]] 2013/09/28 23:06//