diff options
Diffstat (limited to 'doc/man-icecream.7.docbook')
| -rw-r--r-- | doc/man-icecream.7.docbook | 312 |
1 files changed, 312 insertions, 0 deletions
diff --git a/doc/man-icecream.7.docbook b/doc/man-icecream.7.docbook new file mode 100644 index 0000000..759961d --- /dev/null +++ b/doc/man-icecream.7.docbook @@ -0,0 +1,312 @@ +<?xml version="1.0" ?> +<!-- vim:set ts=4 noet syntax=xml: --> +<!DOCTYPE refentry PUBLIC "-//KDE//DTD DocBook XML V4.2-Based Variant V1.1//EN" "dtd/kdex.dtd" [ +<!ENTITY % English "INCLUDE"> +]> + +<refentry lang="&language;"> +<refentryinfo> + <title>Icecream User's Manual</title> + <author> + <personname> + <firstname>Cornelius</firstname> + <surname>Schumacher</surname> + </personname> + </author> + <date>April 21th, 2005</date> + <productname>Icecream</productname> +</refentryinfo> + +<refmeta> + <refentrytitle>Icecream</refentrytitle> + <manvolnum>7</manvolnum> +</refmeta> + +<refnamediv> + <refname>Icecream</refname> + <refpurpose>A distributed compile system</refpurpose> +</refnamediv> + +<refsect1> +<title>Description</title> + +<para>Icecream is a distributed compile system for C and C++.</para> + +<para>Icecream is created by SUSE and is based on ideas and code by distcc. Like +distcc it takes compile jobs from your (KDE) build and distributes it to remote +machines allowing a parallel build on several machines you've got. But unlike +distcc Icecream uses a central server that schedules the compile jobs to the +fastest free server and is as this dynamic. This advantage pays off mostly for +shared computers, if you're the only user on x machines, you have full control +over them anyway.</para> + +</refsect1> + +<refsect1> +<title>How to use icecream</title> + +<para>You need:</para> +<itemizedlist> +<listitem> + <para>One machine that runs the scheduler ("./scheduler -d")</para> +</listitem> +<listitem> + <para>Many machines that run the daemon ("./iceccd -d")</para> +</listitem> +</itemizedlist> + +<para>If you want to compile using icecream, make sure $prefix/bin is the first +first entry in your path, e.g. type +<command>export PATH=/opt/icecream/bin:$PATH</command> +(Hint: put this in ~/.bashrc or /etc/profile to not have to type it in +everytime) +</para> + +<para>Then you just compile with <command>make -j <num></command>, where +<num> is the amount of jobs you want to compile in parallel. Don't +exaggerate. Numbers greater than 15 normally cause trouble.</para> + +<para>WARNING: Never use icecream in untrusted environments. Run the deamons and +the scheduler as unpriviliged user in such networks if you have to! But you will +have to rely on homogeneous networks then (see below).</para> + +<para>If you want funny stats, you might want to run "icemon".</para> +</refsect1> + +<refsect1> +<title>Using icecream in heterogeneous environments</title> + +<para>If you are running icecream daemons (note: they _all_ must be running as +root. In the future icecream might gain the ability to know when machines can't +accept a different env, but for now it is all or nothing ) in the same icecream +network but on machines with incompatible compiler versions you have to tell +icecream which environment you are using. Use <command>icecc --build-native</command> to +create an archive file containing all the files necessary to setup the compiler +environment. The file will have a random unique name like +"ddaea39ca1a7c88522b185eca04da2d8.tar.bz2" per default. Rename it to something +more expressive for your convenience, e.g. "i386-3.3.1.tar.bz2". Set +<command>ICECC_VERSION=<filename_of_archive_containing_your_environment></command> +in the shell environment where you start the compile jobs and the file will be +transfered to the daemons where your compile jobs run and installed to a chroot +environment for executing the compile jobs in the environment fitting to the +environment of the client. This requires that the icecream deamon runs as root. +</para> + +<para>If you do not set ICECC_VERSION, the client will use a tar ball provided +by the daemon running on the same machine. So you can always be sure you're not +tricked by incompatible gcc versions - and you can share your computer with +users of other distributions (or different versions of your beloved SUSE +Linux :)</para> +</refsect1> + +<refsect1> +<title>Cross-Compiling using icecream</title> + +<para>SUSE got quite some good machines not having a processor from Intel or +AMD, so icecream is pretty good in using cross-compiler environments similiar +to the above way of spreading compilers. There the ICECC_VERSION varaible looks +like <native_filename>(,<platform>:<cross_compiler_filename>)*, +for example like this: +<literal>/work/9.1-i386.tar.bz2,ia64:/work/9.1-cross-ia64.tar.bz2</literal> +</para> + +<para>How to package such a cross compiler is pretty straightforward if you look +what's inside the tarballs generated by <command>icecc --build-native</command>.</para> + +</refsect1> + +<refsect1> +<title>Cross-Compiling for embedded targets using icecream</title> + +<para>When building for embedded targets like ARM often you'll have a toolchain +that runs on your host and produces code for the target. In these situations you +can exploit the power of icecream as well.</para> + +<para>Create symlinks from where icecc is to the name of your cross compilers +(e.g. arm-linux-g++ and arm-linux-gcc), make sure that these symlinks are in the +path and before the path of your toolchain, with <varname>$ICECC_CC</varname> +and <varname>$ICECC_CXX</varname> you need to tell icecream which compilers to +use for preprocessing and local compiling. e.g. set it to +<command>ICECC_CC=arm-linux-gcc</command> and +<command>ICECC_CXX=arm-linux-g++</command>.</para> + +<para>As the next step you need to create a .tar.bz2 of your cross compiler, +check the result of build-native to see what needs to be present.</para> + +<para>Finally one needs to set <varname>ICECC_VERSION</varname> and point it to +the tar.bz2 you've created. When you start compiling your toolchain will be +used.</para> + +<para>NOTE: with <varname>ICECC_VERSION</varname> you point out on which +platforms your toolchain runs, you do not indicate for which target code will be +generated.</para> + +</refsect1> + +<refsect1> +<title>How to combine icecream with ccache</title> + +<para>The easiest way to use ccache with icecream is putting the symlink +masquerades into /opt/icream/bin and putting small wrapper scripts in +/opt/ccache/bin + +<screen> + cat /opt/ccache/bin/g++: + + #! /bin/sh + + export CCACHE_PATH=/opt/icecream/bin + export PATH=/opt/icecream/bin:/usr/bin:$PATH + ccache g++ "$@" +</screen> +</para> + +<para>Then you can replace /opt/icecream/bin with /opt/ccache/bin in your +<varname>$PATH</varname> and all icecream calls will go through ccache (and Qt +will compile in 62s :)</para> + +<para>Note however that ccache isn't really worth the trouble if you're not +recompiling your KDE three times a day from scratch (it adds quite some overhead +in comparing the preprocessor output and uses quite some disc space and I found +a cache hit of 18% a bit too few, so I disabled it again).</para> + +</refsect1> + +<refsect1> +<title>Debug output</title> + +<para>You can use the environment variable <varname>ICECC_DEBUG</varname> to +control if icecream gives debug output or not. Set it to +<literal>debug</literal> to get debug output. The other possible values are +<literal>error</literal>, <literal>warning</literal> and <literal>info</literal> +(the -v option for daemon and scheduler raise the level per -v on the command +line - so use -vvv for full debug).</para> + +</refsect1> + +<refsect1> +<title>Some Numbers</title> + +<para> +Numbers of my test case (some STL C++ genetic algorithm) +<itemizedlist> + <listitem> + <para>g++ on my machine: 1.6s</para> + </listitem> + <listitem> + <para>g++ on fast machine: 1.1s</para> + </listitem> + <listitem> + <para>icecream using my machine as remote machine: 1.9s</para> + </listitem> + <listitem> + <para>icecream using fast machine: 1.8s</para> + </listitem> +</itemizedlist> +</para> + +<para>The icecream overhead is quite huge as you might notice, but the compiler +can't interleave preprocessing with compilation and the file needs to be +read/written once more and in between the file is transfered.</para> + +<para>But even if the other computer is faster, using g++ on my local machine +is faster. If you're (for whatever reason) alone in your network at some point, +you loose all advantages of distributed compiling and only add the overhead. So +icecream got a special case for local compilations (the same special meaning +that localhost got within $DISTCC_HOSTS). This makes compiling on my machine +using icecream down to 1.7s (the overhead is actually less than 0.1s in +average).</para> + +<para>As the scheduler is aware of that meaning, it will prefer your own +computer if it's free and got not less than 70% of the fastest available +computer.</para> + +<para>Keep in mind, that this affects only the first compile job, the second one +is distributed anyway. So if I had to compile two of my files, I would get +<itemizedlist> + <listitem> + <para>g++ -j1 on my machine: 3.2s</para> + </listitem> + <listitem> + <para>g++ -j1 on the fast machine: 2.2s</para> + </listitem> + <listitem> + <para>using icecream -j2 on my machine: max(1.7,1.8)=1.8s</para> + </listitem> + <listitem> + <para>(using icecream -j2 on the other machine: max(1.1,1.8)=1.8s)</para> + </listitem> +</itemizedlist> +</para> + +<para>The math is a bit tricky and depends a lot on the current state of the +compilation network, but make sure you're not blindly assuming make -j2 halfs +your compilation time.</para> + +</refsect1> + +<refsect1> +<title>What is the best environment for icecream</title> + +<para>In most requirements icecream isn't special, e.g. it doesn't matter what +distributed compile system you use, you won't have fun if your nodes are +connected through than less or equal to 10MBit. Note that icecream compresses +input and output files (using lzo), so you can calc with ~1MBit per compile job +- i.e more than make -j10 won't be possible without delays.</para> + +<para>Remember that more machines are only good if you can use massive +parallelization, but you will for sure get the best result if your submitting +machine (the one you called g++ on) will be fast enough to feed the others. +Especially if your project consists of many easy to compile files, the +preprocessing and file IO will be job enough to need a quick machine.</para> + +<para>The scheduler will try to give you the fastest machines available, so even +if you add old machines, they will be used only in exceptional situations, but +still you can have bad luck - the scheduler doesn't know how long a job will +take before it started. So if you have 3 machines and two quick to compile and +one long to compile source file, you're not safe from a choice where everyone +has to wait on the slow machine. Keep that in mind.</para> + +</refsect1> + +<refsect1> +<title>Network setup for Icecream (firewalls)</title> + +<para>A short overview of the ports icecream requires: + <itemizedlist> + <listitem> + <para> TCP/10245 on the daemon computers (required) </para> + </listitem> + <listitem> + <para> TCP/8765 for the the scheduler computer (required) </para> + </listitem> + <listitem> + <para> TCP/8766 for the telnet interface to the scheduler (optional) </para> + </listitem> + <listitem> + <para> UDP/8765 for broadcast to find the scheduler (optional) </para> + </listitem> +</itemizedlist> +</para> + +<para>Note that the SuSEfirewall2 on SUSE < 9.1 got some problems +configuring broadcast. So you might need the -s option for the daemon +in any case there. If the monitor can't find the scheduler, use +USE_SCHEDULER=<host> icemon (or send me a patch :)</para> + +</refsect1> + + +<refsect1> +<title>See Also</title> +<para>icecream, scheduler, iceccd, icemon</para> +</refsect1> + +<refsect1> +<title>Icecream Authors</title> +<para>Stephan Kulow <coolo@suse.de></para> +<para>Michael Matz <matz@suse.de></para> +<para>Cornelius Schumacher <cschum@suse.de></para> +<para>...and various other contributors.</para> +</refsect1> +</refentry> |