python 2.6.7: added missing dependency for bzip2 (without this python-compression...
[openembedded.git] / docs / usermanual / chapters / common_use_cases.xml
1 <?xml version="1.0" encoding="UTF-8"?>
2 <chapter id="chapter_common_use_cases">
3   <title>Common Use-cases/tasks</title>
4
5   <section id="commonuse_new_distro">
6     <title>Creating a new Distribution</title>
7
8     <para>Creating a new distribution is not complicated, however we urge you
9     to try existing distributions first, because it's also very easy to do
10     wrong. The config need to be created in /conf/distro directory. So what
11     has to be inside? <itemizedlist>
12         <listitem>
13           <para><command>DISTRO_VERSION</command> so users will know which
14           version of distribution they use.</para>
15         </listitem>
16
17         <listitem>
18           <para><command>DISTRO_TYPE</command> (release/debug) variable is
19           used in some recipes to enable/disable some features - for example
20           kernel output on screen for "debug" builds.</para>
21         </listitem>
22
23         <listitem>
24           <para>Type of libc used: will it be glibc
25           (<command>TARGET_OS</command> = "linux") or uclibc
26           (<command>TARGET_OS</command> = "linux-uclibc")?</para>
27         </listitem>
28
29         <listitem>
30           <para>Toolchain versions - for example gcc 3.4.4 based distro will
31           have: <screen>
32 PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}gcc-initial:gcc-cross-initial"
33 PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}gcc:gcc-cross"
34 PREFERRED_PROVIDERS += " virtual/${TARGET_PREFIX}g++:gcc-cross"
35
36 PREFERRED_VERSION_binutils = "2.16"
37 PREFERRED_VERSION_binutils-cross = "2.16"
38
39 PREFERRED_VERSION_gcc = "3.4.4"
40 PREFERRED_VERSION_gcc-cross = "3.4.4"
41 PREFERRED_VERSION_gcc-initial-cross = "3.4.4"
42             </screen></para>
43         </listitem>
44
45         <listitem>
46           <para><command>DISTRO_FEATURES</command> which describe which
47           features distro has. More about it in <link
48           linkend="task-base">task-base</link> section.</para>
49         </listitem>
50
51         <listitem>
52           <para>Versions of kernels used for supported devices: <screen>
53 PREFERRED_VERSION_linux-omap1_omap5912osk ?= "2.6.18+git"
54 PREFERRED_VERSION_linux-openzaurus ?= "2.6.17"
55             </screen></para>
56         </listitem>
57
58         <listitem>
59           <para>To get more stable build it is good to make use of
60           sane-srcdates.inc file which contain working SRCDATE for many of
61           floating recipes. <screen>
62 require conf/distro/include/sane-srcdates.inc
63             </screen> It also should have global <command>SRCDATE</command>
64           value set (format is ISO date: YYYYMMDD): <screen>
65 SRCDATE = "20061014"
66             </screen></para>
67         </listitem>
68       </itemizedlist></para>
69   </section>
70
71   <section id="commonuse_new_machine">
72     <title>Adding a new Machine</title>
73
74     <para>To be able to build for device OpenEmbedded have to know it, so
75     machine config file need to be written. All those configs are stored in
76     /conf/machine/ directory.</para>
77
78     <para>As usual some variables are required: <itemizedlist>
79         <listitem>
80           <para><command>TARGET_ARCH</command> which describe which CPU
81           architecture does machine use.</para>
82         </listitem>
83
84         <listitem>
85           <para><command>MACHINE_FEATURES</command> which describe which
86           features device has. More about it in <link
87           linkend="task-base">task-base</link> section.</para>
88         </listitem>
89
90         <listitem>
91           <para><command>PREFERRED_PROVIDER_virtual/kernel</command> has to
92           point into proper kernel recipe for this machine.</para>
93         </listitem>
94       </itemizedlist></para>
95
96     <para>Next kernel recipe needs to be added.</para>
97   </section>
98
99   <section id="commonuse_new_package">
100     <title>Adding a new Package</title>
101
102     <para>This section is a stub, help us by expanding it.  Learn by example, go through the
103     recipes that are already there and mimic them to do what you want.</para>
104
105     <section>
106         <title>building from unstable source code</title>
107         <para>Building against the latest, bleeding-edge source has some intricacies of its own.
108         For one, it is desirable to pin down a 1 code revision that is known to build to
109         prevent random breakage in OE at the most inopportune time for all OE users.  Here is
110         how to do that properly.
111           <itemizedlist>
112             <listitem><para>for svn: add 'PV = "1.1+svnr${SRCREV}"' to your bb file.</para></listitem>
113             <listitem><para>for cvs: add 'PV = "1.1+cvs${SRCREV}"' to your bb file.</para></listitem>
114           </itemizedlist>
115         Accompany either with an entry to conf/distro/include/sane-srcrevs.inc for a revision that you know
116         builds successfully.
117         </para>
118         <para>
119         If you really absolutely have to follow the latest commits, you can do that by adding
120         'SRCREV_pn-linux-davinci ?= ${AUTOREV}' to your local.conf, for example.  In this case,
121         you'd build against the most recent and unstable source for the pn-linux-davinci package.
122         </para>
123     </section>
124   </section>
125
126   <section id="commonuse_new_image">
127     <title>Creating your own image</title>
128
129     <para>Creating own image is easy - only few variables needs to be set:
130     <itemizedlist>
131         <listitem>
132           <para><command>IMAGE_BASENAME</command> to give a name for your own
133           image</para>
134         </listitem>
135
136         <listitem>
137           <para><command>PACKAGE_INSTALL</command> to give a list of packages
138           to install into the image</para>
139         </listitem>
140
141         <listitem>
142           <para><command>RDEPENDS</command> to give a list of recipes which
143           are needed to be built to create this image</para>
144         </listitem>
145
146         <listitem>
147           <para><command>IMAGE_LINGUAS</command> is an optional list of
148           languages which has to be installed into the image</para>
149         </listitem>
150       </itemizedlist> Then adding of the <emphasis>image</emphasis> class use:
151     <screen>
152 inherit image
153 </screen> And the image recipe is ready for usage.</para>
154   </section>
155
156   <section id="commonuse_prebuilt_toolchain">
157     <title>Using a prebuilt toolchain to create your packages</title>
158
159     <para>It might be necessary to integrate a prebuilt toolchain and other
160     libraries but still be use OpenEmbedded to build packages. One of many
161     approaches is shown and discussed here.</para>
162
163     <section>
164       <title>The toolchain</title>
165
166       <para>We assume the toolchain provides a C and C++ compiler, an
167       assembler and other tools to build packages. The list below shows a gcc
168       3.4.4 toolchain for ARM architectures using glibc. We assume that the
169       toolchain is in your <command>PATH</command>.</para>
170
171       <screen>
172 $ <command>ls</command> pre-built/cross/bin
173
174 arm-linux-g++
175 arm-linux-ld
176 arm-linux-ranlib
177 arm-linux-ar
178 arm-linux-g77
179 arm-linux-readelf
180 arm-linux-as
181 arm-linux-gcc
182 arm-linux-gcc-3.4.4
183 arm-linux-c++
184 arm-linux-size
185 arm-linux-c++filt
186 arm-linux-nm
187 arm-linux-strings
188 arm-linux-cpp
189 arm-linux-objcopy
190 arm-linux-strip
191 arm-linux-objdump
192 </screen>
193     </section>
194
195     <section>
196       <title>The prebuilt libraries</title>
197
198       <para>We need the header files and the libraries itself. The following
199       directory layout is assumed. <command>PRE_BUILT</command> has two
200       subdirectories one is called <emphasis>include</emphasis> and holds the
201       header files and the other directory is called <emphasis>lib</emphasis>
202       and holds the shared and static libraries. Additionally a Qt2 directory
203       is present having a <emphasis>include</emphasis> and
204       <emphasis>lib</emphasis> sub-directory.</para>
205
206       <screen>
207 $ <command>ls</command> $PRE_BUILT
208 include
209 lib
210 qt2
211 </screen>
212     </section>
213
214     <section>
215       <title>Setting up OpenEmbedded</title>
216
217       <para>OpenEmbedded will be setup here. We assume that your machine and
218       distribution is not part of OpenEmbedded and they will be created ad-hoc
219       in the <emphasis>local.conf</emphasis> file. You will need to have
220       <application>BitBake</application> and a current OpenEmbedded version
221       available.</para>
222
223       <section>
224         <title>Sourceable script</title>
225
226         <para>To ease the usage of OpenEmbedded we start by creating a
227         source-able script. This is actually a small variation from the
228         already seen script. We will name it <emphasis>build_source</emphasis>
229         and you will need to source it.</para>
230
231         <screen>
232 BITBAKE_PATH=/where/is/bitbake/bin
233 TOOLCHAIN=/where/is/toolchain/bin
234 HOST_TOOLS=/where/is/hosttools/bin
235 export PRE_BUILT=/where/is/pre-built
236
237 export PATH=$BITBAKE_PATH:$TOOLCHAIN:$HOST_TOOLS:$PATH
238 export OEDIR=$PWD
239 export LOCALDIR=$PWD/secret-isv
240                     </screen>
241
242         <para>Use <command>source build_source</command> to source the script,
243         use <command>env</command> to check that the variable where
244         exported.</para>
245       </section>
246
247       <section>
248         <title>Creating the local.conf</title>
249
250         <para>We will configure OpenEmbedded now, it is very similar to what
251         we have done above.</para>
252
253         <screen>
254 DL_DIR = "${OEDIR}/sources"
255 BBFILES := "${OEDIR}/openembedded/recipes/*/*.bb ${LOCALDIR}/recipes/*/*.bb"
256 BBFILE_COLLECTIONS = "upstream local"
257 BBFILE_PATTERN_upstream = "^${OEDIR}/openembedded/recipes/"
258 BBFILE_PATTERN_local = "^${LOCALDIR}/recipes/"
259 BBFILE_PRIORITY_upstream = "5"
260 BBFILE_PRIORITY_local = "10"
261 BBMASK = ""
262                     </screen>
263
264         <para>${OEDIR}/openembedded will be a upstream release of
265         OpenEmbedded. Above we have assumed it is in the current working
266         directory. Additionally we have a ${LOCALDIR}, we combine these two
267         directories as a special <link linkend="collections">BitBake
268         Collection</link>.</para>
269
270         <screen>
271 #
272 # machine stuff
273 #
274 MACHINE = "secret-killer"
275 PACKAGE_EXTRA_ARCHS = "armv4 armv4t armv5te iwmmxt xscale""
276 TARGET_CC_ARCH = "-mcpu=xscale -mtune=iwmmxt"
277 TARGET_ARCH = "arm"
278 PACKAGE_ARCH="xscale"
279                 </screen>
280
281         <para>We tell OpenEmbedded that we build for the ARM platform and
282         optimize for xscale and iwmmxt.</para>
283
284         <screen>
285 INHERIT += " package_ipk debian"
286 TARGET_OS  = "linux"
287 TARGET_FPU = "soft"
288 DISTRO = "secret-disro"
289 DISTRO_NAME = "secret-distro"
290 DISTRO_VERSION = "x.y.z"
291 DISTRO_TYPE = "release"
292                 </screen>
293
294         <para>Create a distribution ad-hoc as well. We tell OpenEmbedded that
295         we build for linux and glibc using soft float as fpu. If your
296         toolchain is a uclibc toolchain you will need to set
297         <command>TARGET_OS</command> to linux-uclibc.</para>
298
299         <screen>
300 export CC="${CCACHE}arm-linux-gcc-3.4.4 ${HOST_CC_ARCH}"
301 export CXX="${CCACHE}arm-linux-g++ ${HOST_CC_ARCH}"
302 export CPP="arm-linux-gcc-3.4.4 -E"
303 export LD="arm-linux-ld"
304 export AR="arm-linux-ar"
305 export AS="arm-linux-as"
306 export RANLIB="arm-linux-ranlib"
307 export STRIP="arm-linux-strip"
308                 </screen>
309
310         <para>The above variables replace the ones from
311         <emphasis>bitbake.conf</emphasis>. This will make OpenEmbedded use the
312         prebuilt toolchain.</para>
313
314         <screen>
315 #
316 # point OE to the lib and include directory
317 #
318 TARGET_CPPFLAGS_append = " -I${PRE_BUILT}/include "
319 TARGET_LDFLAGS_prepend = " -L${PRE_BUILT}/qt2/lib -L${PRE_BUILT}/lib \
320 -Wl,-rpath-link,${PRE_BUILT}/lib -Wl,-rpath-link,${PRE_BUILT}/qt2/lib "
321
322 # special to Qt/Qtopia
323 QTDIR  = "${PRE_BUILT}/qt2"
324 QPEDIR = "${PRE_BUILT}"
325 palmtopdir = "/opt/Qtopia"
326 palmqtdir  = "/opt/Qtopia"
327                 </screen>
328
329         <para>We will add the <command>PRE_BUILT</command> libraries to the
330         include and library paths. And the same is done for the special
331         version of Qt we have in your <command>PRE_BUILT</command>
332         directory.</para>
333
334         <screen>
335 ASSUME_PROVIDED += " virtual/${TARGET_PREFIX}gcc "
336 ASSUME_PROVIDED += " virtual/libc "
337 ASSUME_PROVIDED += " virtual/qte "
338 ASSUME_PROVIDED += " virtual/libqpe "
339 ASSUME_PROVIDED += " libqpe-opie "
340                 </screen>
341
342         <para>Now we have told <application>BitBake</application> that the C
343         library, compiler and Qtopia is already provided. These lines will
344         avoid building binutils, gcc initial, glibc, gcc.</para>
345
346         <screen>
347 <command>source</command> build_source
348 <command>bitbake</command> your-killer-app
349                 </screen>
350
351         <para>You should be able to create the packages you want to using the
352         prebuilt toolchain now.</para>
353       </section>
354     </section>
355
356     <section>
357       <title>Useful hints</title>
358
359       <para>If you have more prebuilt libraries you need to add additional
360       <command>ASSUME_PROVIDED</command> lines to your
361       <emphasis>local.conf</emphasis>. Using <command>bitbake -vvv
362       PACKAGE</command> you can easily see the package names you could
363       <command>ASSUME_PROVIDED</command> if you have some prebuilt.</para>
364     </section>
365
366     <section>
367       <title>Issues with this approach</title>
368
369       <screen>
370 NOTE: Couldn't find shared library provider for libqtopia.so.1
371 NOTE: Couldn't find shared library provider for libqtopia2.so.2
372 NOTE: Couldn't find shared library provider for libqpe.so.1
373 NOTE: Couldn't find shared library provider for libpthread.so.0
374 NOTE: Couldn't find shared library provider for libstdc++.so.6
375 NOTE: Couldn't find shared library provider for libqte.so.2
376 NOTE: Couldn't find shared library provider for libgcc_s.so.1
377 NOTE: Couldn't find shared library provider for libc.so.6
378 NOTE: Couldn't find shared library provider for libm.so.6
379 </screen>
380
381       <para>OpenEmbedded tries to automatically add run-time dependencies
382       (RDEPENDS) to generated packages. It is inspecting binaries and
383       libraries and uses the <emphasis><link linkend="shlibs">shlibs</link>
384       </emphasis> system to do add dependencies for the linked libraries,
385       however in this case it was not able to find packages providing these
386       libraries as they were prebuilt.
387       </para>
388
389       <para>One way to resolve this problem is to provide an explicit mapping
390       using the ASSUME_SHLIBS variable in a config file <filename>local.conf</filename>.
391       For example, for the libraries above (partial):
392       <screen>
393 ASSUME_SHLIBS = "libqtopia2.so.2:qtopia2_2.4 libc.so.6:libc"
394 </screen>
395       The format is shlib_file_name:package[_version]. If a version is specified it will be
396       used as the minimal (>=) version for the dependency.</para>
397     </section>
398   </section>
399
400   <section id="commonuse_new_package_format">
401     <title>Using a new package format</title>
402
403     <para>This section is a stub, help us by expanding it</para>
404   </section>
405
406   <section id="commonuse_qte_sdk">
407     <title>Creating Software Development Kits (SDKs)</title>
408
409     <section>
410         <title>What is provided by a SDK</title>
411
412         <para>The Software Development Kit (SDK) should be easy to install and
413         enable your user-base to create binaries and libraries that work on the
414         target hardware.
415         </para>
416
417         <para>To accomplish this goal OpenEmbedded SDKs contain tools for the
418         host and tools for the target hardware. Among these tools is a cross
419         compiler, libraries and header files for additional dependencies, pkg-config
420         files to allow buildsystems to easily find the dependencies, a file with
421         results for autoconf and a script that can be sourced to setup the
422         environment.
423         </para>
424     </section>
425
426     <section>
427         <title>Creating a SDK with your libraries pre-installed</title>
428
429         <section>
430             <title>Preparing the host side</title>
431             <para>Your SDK might need utilities that will run on the
432             host. These could include scripts, buildsystem software like
433             cmake, or an emulator like qemu. For these dependencies it is
434             imported that they <emphasis>inherit sdk</emphasis> and by
435             convention end with <emphasis>-sdk</emphasis> in the
436             <command>PN</command>.
437             </para>
438
439             <para>A new task should be created that will assure that all
440             host utilities will be installed. Place a file called
441             <filename>task-YOUR-toolchain-host.bb</filename> in the
442             <filename>recipes/tasks</filename> directory and place the
443             following content in it:
444 <screen>
445 require task-sdk-host.bb
446 DESCRIPTION = "Host packages for YOUR SDK"
447 LICENSE = "MIT"
448 ALLOW_EMPTY = "1"
449 RDEPENDS_${PN} += "YOUR-DEPENDENCY-sdk"
450 </screen>
451             </para>
452         </section>
453
454         <section>
455             <title>Preparing the target side</title>
456             <para>Your SDK should provide your user with header files and libraries
457             he will need when doing application development. In OpenEmbedded the
458             <command>${PN}-dev</command> is providing the header files, pkg-config
459             files and symbolic links to libraries to allow using the library. The SDK
460             should install these development packages to the SDK.
461             </para>
462
463             <para>To install the development packages you will need to create a
464             new task. Create a new file <filename>task-YOUR-toolchain-target.bb</filename>
465             in the <filename>recipes/tasks</filename> directory and place the
466             following content in it:
467 <screen>
468 DESCRIPTION = "Target package for YOUR SDK"
469 LICENSE = "MIT"
470 ALLOW_EMPTY = "1"
471
472 PR = "r0"
473
474 RDEPENDS_${PN} += "\
475         task-sdk-bare \
476         your-lib-dev \
477         your-data
478         "
479 </screen>
480             </para>
481         </section>
482
483         <section>
484             <title>Putting it together</title>
485             <para>In the previous two sections we have prepared the host and
486             target side. One thing that is missing is combining the two newly
487             created tasks and actually create the SDK. This is what we are going
488             to do now.</para>
489
490             <para>Create <filename>meta-toolchain-YOU.bb</filename> in the
491             <filename>recipes/meta</filename> directory and place the following
492             content in it:
493 <screen>
494 PR = "r0"
495 TOOLCHAIN_TARGET_TASK = "task-YOUR-toolchain-target"
496 TOOLCHAIN_HOST_TASK = "task-YOUR-toolchain-host"
497
498 require meta-toolchain.bb
499 SDK_SUFFIX = "toolchain-YOUR"
500 </screen>
501
502             </para>
503
504             <para>Using <command>bitbake meta-toolchain-YOU</command> the SDK
505             creation should be started and you should find a <filename>sdk</filename>
506             directory inside your deploy directory with a SDK waiting for you. With
507             the above command you still need to have OE configured with your
508             <filename>conf/local.conf</filename> to select the machine and
509             distribution you are targeting.
510             </para>
511
512             <note><para>SDK creation currently does not work with the <emphasis>DISTRO</emphasis>
513             set to <emphasis>micro</emphasis>.</para></note>
514
515             <note><para>If the environment-setup script packaged in the SDK should
516             require more environment look at the <filename>meta-toolchain-qte.bb</filename>
517             to accomplish this.</para></note>
518         </section>
519     </section>
520   </section>
521
522   <section>
523     <title>Creating and Using a Qt Embedded SDK</title>
524
525         <section>
526             <title>Creating the SDK</title>
527
528             <para>The SDK should contain a build of Qt Embedded, but also
529             optional dependencies like directFB, glib-2.0, gstreamer-0.10, tslib
530             and more esoteric dependencies like mysql and postgres. This allows
531             developers to simply start developing using Qt and enables system
532             integrator to easily recompile Qt and base libraries without tracking
533             down extra dependencies.
534             </para>
535
536             <para>OpenEmbedded provides an easy way to create a Qt Embedded
537             SDK. In
538             <filename>recipes/tasks/task-qte-toolchain-host.bb</filename> host
539             tools like moc, uic, rcc, qmake will get installed and in <filename>
540             recipes/tasks/task-qte-toolchain-target.bb</filename> the Qt4 header
541             files and libraries will be installed.
542             </para>
543
544             <para>To build the SDK, setup OpenEmbedded in the usual way by picking
545             a <emphasis>DISTRO</emphasis> and <emphasis>MACHINE</emphasis>. Issue
546             the below command and after the operation finished you should find
547             a SDK in the deployment directory.
548 <screen>
549 $ <command>bitbake</command> meta-toolchain-qte
550 </screen>
551             </para>
552
553             <note><para>The deployment directory depends on the distribution
554             and used C library. In the case of Angstrom and glibc it is
555             located in <filename>tmp/deploy/glibc/sdk</filename>.</para></note>
556
557             <note><para>Change <filename>qt4-embedded.inc</filename> and
558             <filename>qt4.inc</filename> for using different Qt configuration
559             flags. This might include a custom qconfig.h to produce a reduced
560             size build.</para></note>
561
562             <note><para>When distributing the SDK make sure to include a written offer
563             to provide the sourcecode of GPL licensed applications or provide
564             parts of the <filename>sources</filename> folder. The <filename>
565             sources</filename> folder is located right next to the <filename>sdk</filename>
566             one.</para></note>
567         </section>
568
569
570         <section>
571             <title>Using the Qt Embedded SDK</title>
572
573             <para>In this example we are assuming that the target hardware
574             is an armv5t system and the SDK targets the Angstrom Distribution. You
575             should start by downloading the SDK and untar it to the root folder
576             (<filename>/</filename>). Once this operation is finished you will
577             find a new directory <filename>/usr/local/angstrom/arm/</filename> and
578             it contains the <filename>environment-setup</filename> to setup the
579             <emphasis>QMAKESPEC</emphasis> and various other paths.
580             </para>
581
582 <screen>
583 Untar the SDK once
584 $ <command>tar</command> -C / -xjf angstrom-armv5te-linux-gnueabi-toolchain-qte.tar.bz2
585
586 Before using it source the environment
587 $ <command>.</command> /usr/local/angstrom/arm/environment-setup
588
589 Use qmake2 to build software for the target
590 $ <command>qmake2</command>
591 </screen>
592
593             <para>Creating and building a simple example. We will create a simple
594             Qt Embedded application and use <command>qmake2</command> and
595             <command>make</command> to cross compile.
596
597 <screen>
598 $ <command>.</command> /usr/local/angstrom/arm/environment-setup
599 $ <command>cd</command> $HOME
600 $ <command>mkdir</command> qte-example
601 $ <command>cd</command> qte-example
602
603 $ <command>echo</command> "TEMPLATE=app
604 SOURCES=main.cpp
605 " > qte-example.pro
606
607 $ <command>echo</command> '#include &lt;QApplication&gt;
608 #include &lt;QPushButton&gt;
609
610 int main(int argc, char** argv) {
611     QApplication app(argc, argv);
612
613     QPushButton btn("Hello World");
614     btn.show();
615     btn.showMaximized();
616
617     return app.exec();
618 }
619 ' > main.cpp
620
621 $ <command>qmake2</command>
622 $ <command>make</command>
623 </screen>
624             </para>
625
626         </section>
627   </section>
628 </chapter>