bitbake.git
8 years agoRelease 1.12.0 1.12.0
Richard Purdie [Fri, 18 Feb 2011 17:03:42 +0000 (17:03 +0000)]
Release 1.12.0

8 years agouihelper: import bb.build, kill commented lines
Chris Larson [Wed, 16 Feb 2011 22:58:07 +0000 (15:58 -0700)]
uihelper: import bb.build, kill commented lines

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agocooker: don't choke if we have nothing to parse
Chris Larson [Wed, 16 Feb 2011 22:50:50 +0000 (17:50 -0500)]
cooker: don't choke if we have nothing to parse

If all our recipes were cached, there's no reason to fire off any parsing
progress events at all.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agosiggen: add bb.data, bb.parse imports
Chris Larson [Wed, 16 Feb 2011 22:41:37 +0000 (15:41 -0700)]
siggen: add bb.data, bb.parse imports

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agorunqueue: simplify RunQueueStats.copy
Chris Larson [Wed, 16 Feb 2011 22:24:24 +0000 (15:24 -0700)]
runqueue: simplify RunQueueStats.copy

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agorunqueue: pass a copy of the RunQueueStats to events
Chris Larson [Wed, 16 Feb 2011 22:18:06 +0000 (15:18 -0700)]
runqueue: pass a copy of the RunQueueStats to events

This avoids cases where the stats are modified after the event is fired but
before it's dispatched to the UI.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoserver: clean up call of write_profile_stats
Chris Larson [Wed, 16 Feb 2011 00:07:00 +0000 (17:07 -0700)]
server: clean up call of write_profile_stats

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoserver: add profiling support to ProcessServer
Chris Larson [Tue, 15 Feb 2011 23:59:56 +0000 (16:59 -0700)]
server: add profiling support to ProcessServer

Based on poky's support, particularly the format of the processed log file and
the log message shown to the user regarding the files written.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoEnable some DeprecationWarnings
Chris Larson [Sat, 12 Feb 2011 04:26:51 +0000 (21:26 -0700)]
Enable some DeprecationWarnings

We'll be skipping the Pending Deprecation step given our release process.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agopersist_data: handle lock in sqlite and use the shared cache
Chris Larson [Thu, 10 Feb 2011 00:56:42 +0000 (17:56 -0700)]
persist_data: handle lock in sqlite and use the shared cache

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agopersist_data: drop SQLData indirection
Chris Larson [Fri, 11 Feb 2011 16:42:38 +0000 (09:42 -0700)]
persist_data: drop SQLData indirection

The common case (if not the only case) is to only use a single domain.  The
only reason SQLData exists is to make it easier to delete a domain.  Yet,
there's no need for us to delete a domain if SQLTable knows how to clear
itself out.  So, add clear() to the table and pass the domain to persist().

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agopersist_data: use better items/values methods for SQLTable
Chris Larson [Thu, 10 Feb 2011 00:57:35 +0000 (17:57 -0700)]
persist_data: use better items/values methods for SQLTable

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agofetch2: Correct the clean() mechanism for the fetcher2 code
Saul Wold [Wed, 9 Feb 2011 22:30:29 +0000 (14:30 -0800)]
fetch2: Correct the clean() mechanism for the fetcher2 code

This create a clean() method in each of the fetcher modules
and correctly cleans the .done stamp file and lock files

(From Poky rev: 14dea89521c0c648e8e543388096a6dcd6d4f2e0)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Write mirror tarballs if enabled and they don't exist, also set...
Richard Purdie [Thu, 10 Feb 2011 15:14:49 +0000 (15:14 +0000)]
bitbake/fetch2/git: Write mirror tarballs if enabled and they don't exist, also set a default value for mirror tarball generation

(From Poky rev: 59a08262aba2d2b9b8b299a2818fb4cfe13b9909)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLY
Richard Purdie [Wed, 9 Feb 2011 01:23:56 +0000 (01:23 +0000)]
bitbake/fetch2: Fix negated if check for BB_FETCH_PREMIRRORONLY

(From Poky rev: 29ed2c6e9a0c3cc57c5bbaf3c53e1cff8586c51c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2: Add SRPM knowledge
Mark Hatle [Tue, 8 Feb 2011 00:18:18 +0000 (18:18 -0600)]
fetch2: Add SRPM knowledge

Enable the fetcher to be able to unpack and SRPM.  By default the system will
unpack the contents of the SRPM into the WORKDIR.

A new syntax "unpack=file" was developed for the SRC_URI, to allow for a
recipe to extract a specific file within an SRPM.  An unpack operation will
then be executed on the extracted file.

In order to apply extracted patches (or unpack files not specified with
unpack), you must specify the path using WORKDIR, i.e.:

file://${WORKDIR}/mypatch.patch

(From Poky rev: 906285ff00d6ffd3fd7713af52250e7c6503edb7)

Signed-off-by: Mark Hatle <mark.hatle@windriver.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirror
Richard Purdie [Tue, 8 Feb 2011 17:30:45 +0000 (17:30 +0000)]
bitbake/fetch2: Ignore UnboundLocalError in exception handler in try_mirror

(From Poky rev: aa45760702e874977454778659c205b29d1ff049)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Move symlink handling into try_mirror where is belongs instead of...
Richard Purdie [Tue, 8 Feb 2011 12:46:25 +0000 (12:46 +0000)]
bitbake/fetch2: Move symlink handling into try_mirror where is belongs instead of the main download function

(From Poky rev: ee1a9c0476cc4b2ce9dfb0faa29a1371a8517c40)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after premirrors...
Richard Purdie [Tue, 8 Feb 2011 12:44:06 +0000 (12:44 +0000)]
bitbake/fetch2: When using BB_FETCH_PREMIRRORONLY, set BB_NO_NETWORK after premirrors as there could be data processing needed by the real fetcher

(From Poky rev: c4e66a7fe0c314b843aac6340995c584ec156529)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Ensure we only remove files, not directories when fetch failures...
Richard Purdie [Tue, 8 Feb 2011 12:42:24 +0000 (12:42 +0000)]
bitbake/fetch2: Ensure we only remove files, not directories when fetch failures occur

(From Poky rev: 91eeb9b0d334275ff2730e55e86fd351e1b652ef)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Ensure original ud is preserved in try_mirror
Richard Purdie [Tue, 8 Feb 2011 12:40:40 +0000 (12:40 +0000)]
bitbake/fetch2: Ensure original ud is preserved in try_mirror

(From Poky rev: 0713fb6b99486cd08c6036dba7aeb15e3bfb8129)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Fix broken variable reference
Richard Purdie [Tue, 8 Feb 2011 09:33:52 +0000 (09:33 +0000)]
bitbake/fetch2/git: Fix broken variable reference

(From Poky rev: a235c67479846590eb2c7236bec3360b23c3e9b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Ensure unresolved branches are translated into revisions
Richard Purdie [Tue, 8 Feb 2011 09:27:59 +0000 (09:27 +0000)]
bitbake/fetch2/git: Ensure unresolved branches are translated into revisions

(From Poky rev: 5920e85c561624e657c126df58f5c378a8950bbc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2: add try/finally to ensure lockfile is unlocked on failure
Saul Wold [Tue, 8 Feb 2011 01:29:46 +0000 (17:29 -0800)]
fetch2: add try/finally to ensure lockfile is unlocked on failure

(From Poky rev: 1544aa8ab4a80d529a001e27b473645f2caec87c)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2: add runfetchcmd to import for fetchers
Saul Wold [Mon, 7 Feb 2011 23:42:45 +0000 (15:42 -0800)]
fetch2: add runfetchcmd to import for fetchers

(From Poky rev: 232b6f3c92928c333ad1201aa8eb3706e7251cdf)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2:Fetch Make using the fn based cache optional
Richard Purdie [Mon, 7 Feb 2011 21:12:51 +0000 (21:12 +0000)]
bitbake/fetch2:Fetch Make using the fn based cache optional

(From Poky rev: 500c66337c7cb5e3044a02ef761097713e47f523)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Fix pickling issues with fetcher exceptions
Richard Purdie [Mon, 7 Feb 2011 20:46:42 +0000 (20:46 +0000)]
bitbake/fetch2: Fix pickling issues with fetcher exceptions

See the problems in http://bugs.python.org/issue1692335, need to set self.args
correctly.

(From Poky rev: d4b4b48addfa781d7b94965e0477974c3fb6dbb3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs
Richard Purdie [Mon, 7 Feb 2011 15:28:05 +0000 (15:28 +0000)]
bitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs

(From Poky rev: 94faffdaf6c13ce59987aab28383d66a9a0bf100)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug...
Richard Purdie [Mon, 7 Feb 2011 12:08:32 +0000 (12:08 +0000)]
bitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug fix the code

(From Poky rev: 37624b97450f2ba3d6fad3e1e51818486451447e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Match Fetcher log domain to that in bb.msg
Richard Purdie [Mon, 7 Feb 2011 14:46:53 +0000 (14:46 +0000)]
bitbake/fetch2: Match Fetcher log domain to that in bb.msg

(From Poky rev: ded83ffc3bd86553c9b540795660985905c925ba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may...
Richard Purdie [Mon, 7 Feb 2011 10:13:29 +0000 (10:13 +0000)]
bitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may not exist

(From Poky rev: 681bcf4e6b606dde2029d143805023a927285917)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for...
Richard Purdie [Mon, 7 Feb 2011 09:34:24 +0000 (09:34 +0000)]
bitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for calling _contains_ref()

(From Poky rev: 330886826770ff6ec1449dc375cb4c3604b2736b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer alway...
Richard Purdie [Mon, 7 Feb 2011 09:13:44 +0000 (09:13 +0000)]
bitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer always generate a mirror tarball and it isn't a good guide to fetcher success

(From Poky rev: 917d3e9697acefe308e7139e86df37a072ee3500)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Use True instead of integer values
Richard Purdie [Fri, 4 Feb 2011 14:40:41 +0000 (14:40 +0000)]
bitbake/fetch2: Use True instead of integer values

(From Poky rev: 7202a77134029cb37540c785ce0161a4dd574853)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Drop name/override ordering backwards compatibility code
Richard Purdie [Fri, 4 Feb 2011 14:37:09 +0000 (14:37 +0000)]
bitbake/fetch2: Drop name/override ordering backwards compatibility code

(From Poky rev: 8f2abf4a9f000d42e98c4936e393bd5033a1af48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/svn: The concept of using dates with svn is totally insane, drop support
Richard Purdie [Fri, 4 Feb 2011 13:46:31 +0000 (13:46 +0000)]
bitbake/fetch2/svn: The concept of using dates with svn is totally insane, drop support

(From Poky rev: ee6f34483a9bfeca5b5c3656fd62bb4849d5fbb8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class
Richard Purdie [Fri, 4 Feb 2011 13:14:03 +0000 (13:14 +0000)]
bitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class

(From Poky rev: ca7adf75295c2a6041b891bfa61e0b4bc2f7c860)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Fix typo
Richard Purdie [Fri, 4 Feb 2011 11:06:16 +0000 (11:06 +0000)]
bitbake/fetch2: Fix typo

(From Poky rev: 08a9fef4fa0036708f29bd150d7ecf66354e5475)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Drop legacy CVSDATE support
Richard Purdie [Fri, 4 Feb 2011 11:03:34 +0000 (11:03 +0000)]
bitbake/fetch2: Drop legacy CVSDATE support

(From Poky rev: 806d1c25f8ca8ab5116652e6f82f7d9ee5774278)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate
Richard Purdie [Fri, 4 Feb 2011 11:02:15 +0000 (11:02 +0000)]
bitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate

(From Poky rev: f60d9d6bd4bb7691b62bf1f4b1fbeaa1faa53879)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in...
Richard Purdie [Fri, 4 Feb 2011 10:59:42 +0000 (10:59 +0000)]
bitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in the FetchMethod class

(From Poky rev: 831c6c1d3c23c50f9c55a6bbf8dc6311bca3881b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Rename Fetch class to FetchMethod
Richard Purdie [Fri, 4 Feb 2011 10:49:27 +0000 (10:49 +0000)]
bitbake/fetch2: Rename Fetch class to FetchMethod

(From Poky rev: ab0dd1397491478ee6149283e5ba8775dd8cdc3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Drop old md5 handling code
Richard Purdie [Fri, 4 Feb 2011 10:30:54 +0000 (10:30 +0000)]
bitbake/fetch2: Drop old md5 handling code

Drop some old md5 functions since we have improved functionality now which includes
sha256 checksum support. This stops each download being md5 checksumed twice.

Also change ".md5" stamp extentions to ".done" to better describe its use as a
download complete marker file and no longer write the md5 sum to the files.

(From Poky rev: 74b71864fed79ce60e721945c8e239b3ebf49200)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Rewrite and improve exception handling, reusing core functions for...
Richard Purdie [Fri, 4 Feb 2011 10:26:21 +0000 (10:26 +0000)]
bitbake/fetch2: Rewrite and improve exception handling, reusing core functions for common operations where possible

(From Poky rev: d08397ba4d1331993300eacbb2f78fcfef19c1cf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException...
Richard Purdie [Thu, 3 Feb 2011 23:23:11 +0000 (23:23 +0000)]
bitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException class

(From Poky rev: f6eefb3ca3bb2a5ea0ec1364bdb0bc41ae58c815)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Drop unused parameter for localpath() and update comments
Richard Purdie [Thu, 3 Feb 2011 22:02:09 +0000 (22:02 +0000)]
bitbake/fetch2: Drop unused parameter for localpath() and update comments

(From Poky rev: 8daab5b95157dda6854fe6bf1929f911fe3cf25e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Simplfy setup_srcrevs
Richard Purdie [Thu, 3 Feb 2011 21:43:01 +0000 (21:43 +0000)]
bitbake/fetch2: Simplfy setup_srcrevs

(From Poky rev: 7e4fbfc1c1887a1a0507b60244aa53b8b1994edd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Simplify localpath variable handling FetchData init
Richard Purdie [Thu, 3 Feb 2011 21:40:03 +0000 (21:40 +0000)]
bitbake/fetch2: Simplify localpath variable handling FetchData init

(From Poky rev: 49a022d25d35115e7286e2ca2530566da2d71aa8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Define a sane localpath function and remove code duplication
Richard Purdie [Thu, 3 Feb 2011 21:20:39 +0000 (21:20 +0000)]
bitbake/fetch2: Define a sane localpath function and remove code duplication

(From Poky rev: 5a52cb80d6d62c704692ad6cfa93426c4a17bc52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Move ud.localfile setup into urldata_init
Richard Purdie [Thu, 3 Feb 2011 21:17:26 +0000 (21:17 +0000)]
bitbake/fetch2: Move ud.localfile setup into urldata_init

(From Poky rev: 972eb5fababb33b5537fcfbbaf8e33ea820f0fee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Drop horrible SRCREV hack which is no longer needed
Richard Purdie [Thu, 3 Feb 2011 20:33:07 +0000 (20:33 +0000)]
bitbake/fetch2: Drop horrible SRCREV hack which is no longer needed

(From Poky rev: 0e5404cedadaea6738ef6468d7eb8a24d23d9bab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Switch to completely use bare clones
Richard Purdie [Sun, 6 Feb 2011 10:20:18 +0000 (10:20 +0000)]
bitbake/fetch2/git: Switch to completely use bare clones

We no longer need index/workdir support in the mirror tree and it causes all
kind of reference naming problems.Simplifying the code to remove this and use
just bare clones addresses this problem.

We increase the "version" number on the mirror tarballs to reflect the change
and ensure older mirror tarballs are not used as they would break.

(From Poky rev: e4c1e99f234491af1c0fc886e73deedaef108a1a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2/git.py: Fix problem when fetching with no origin
Saul Wold [Sat, 5 Feb 2011 22:11:25 +0000 (14:11 -0800)]
fetch2/git.py: Fix problem when fetching with no origin

When updating a git repo, its possible to not have an origin,
so fail gracefully with a debug message.

(From Poky rev: 192f631129e25b959d501c46c3156b7037b49c22)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Ensure we prune origin to remove stale branches
Richard Purdie [Sat, 5 Feb 2011 13:16:45 +0000 (14:16 +0100)]
bitbake/fetch2/git: Ensure we prune origin to remove stale branches

Thanks to Bruce Ashfield for the tip.

(From Poky rev: 96810cade04cad44d70a7c6e51aacdf89caef92b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags...
Richard Purdie [Thu, 3 Feb 2011 20:16:05 +0000 (20:16 +0000)]
bitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags are forced to the upstream revisions and all revisions and tags are always fetched

(From Poky rev: dfc5faf717ac1f917af398097d5295128548a55d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Remove old local file acceleration code
Richard Purdie [Thu, 3 Feb 2011 18:24:25 +0000 (18:24 +0000)]
bitbake/fetch2: Remove old local file acceleration code

Since local mirror fetches are always symlinked from the download directory
directly, there is no need for this premirrors hack which doesn't cover
mirrors and also abuses the localpath variable with inconsistent results.

(From Poky rev: 282a828f3dc373d8f1397827ebbe1be1c54f2d2a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory
Richard Purdie [Thu, 3 Feb 2011 18:22:06 +0000 (18:22 +0000)]
bitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory

When files are fetched from a mirror source that happens to be local,
ensure links are created for the file since subsequent fetch calls
can then follow the links to find files.

Any other approach such as the existing manipulations of localpath
internally to the fetcher are prone to errors, races and other issues.

(From Poky rev: fa9fda05e1d269446b51050195b891346482e8bb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: When encoding a file:// url, drop user and host information
Richard Purdie [Thu, 3 Feb 2011 18:19:23 +0000 (18:19 +0000)]
bitbake/fetch2: When encoding a file:// url, drop user and host information

When processing a cvs SRC_URI to a file:// mirror, the user and host information
will break the mirror processing. This patch addresses it by only constructing
valid urls.

(From Poky rev: 7f99605562119a13a2510a3c990e3cf577ad764e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/lib/bb/fetch2: Assign a default name in the names array as empty strings...
Richard Purdie [Thu, 3 Feb 2011 00:16:23 +0000 (00:16 +0000)]
bitbake/lib/bb/fetch2: Assign a default name in the names array as empty strings as names cause problems for python

(From Poky rev: a55d86b4051f6d063f9a67db026f7da6f9b50cc6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Add missing parameter to localcount_internal_helper
Richard Purdie [Wed, 2 Feb 2011 18:48:13 +0000 (18:48 +0000)]
bitbake/fetch2: Add missing parameter to localcount_internal_helper

(From Poky rev: 08cd6c1cb9639e958e0c60a02e317a22cf43f880)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Ensure target directory exists when copying files
Richard Purdie [Wed, 2 Feb 2011 17:46:37 +0000 (17:46 +0000)]
bitbake/fetch2/git: Ensure target directory exists when copying files

(From Poky rev: adfa6c40dadebb18bfd457859300d8c093b007f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agofetch2: Allow multiple src rev to be used in one src_uri
Yu Ke [Wed, 26 Jan 2011 12:14:06 +0000 (20:14 +0800)]
fetch2: Allow multiple src rev to be used in one src_uri

* SRC_URI format:
the SRC_URI are extended to allow multiple src rev:
name=<name1>,<name2>,...<name-n>
branch=<branch1>,<branch2>,...,<branch-n>
also SRCREV can be defined with
SRCREV_<name1> = xxxxx
SRCREV_<name2> = xxxxx

* FetchData extention
to support multiple src rev, several FetchData data are added:
- FetchData.names: list of name in SRC_URI,  one name per srcrev. name is the index of revision and branch
- FetchData.revisions: dictionary of name->revision.
- FetchData.branches: dictionary of name->branch.

For example, linux-yocto recipes becomes:
SRC_URI = "git://git.pokylinux.org/linux-yocto-2.6.37;protocol=git;branch=${KBRANCH},meta;name=machine,meta"

FetchData.names = ['machine', 'meta']
FetchData.revisions = { 'machine':xxxxx, 'meta':xxxxxx }
FetchData.branches = { 'machine':${KBRANCH}, 'meta':'meta'}

* generic revision handling extension
the related revision handling code in fetch2.__init__.py are changed accordingly. the major change is add name parameter to indicate which src rev to handling. originally there is one src rev per FetchData, so FetchData parameter is enough. now since one FetchData has multiple src rev, it is necessary to use FetchData + name to specifiy src rev.

* git extension
git fetcher are also revised to take advantage of the multiple src rev in FetchData. especially the download() method are enhanced to fetch multiple src rev.

* other fetcher (svn, hg, ...) does not support multiple src rev. they just sync the API to add name, and then simply ignore the name. no actually functional change

(From Poky rev: 0e837e6844be449659bb96a1498e54a8b9442d13)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agogit.py: Remove the source tree tarball archives
Yu Ke [Tue, 18 Jan 2011 16:58:42 +0000 (00:58 +0800)]
git.py: Remove the source tree tarball archives

Since we're now always providing the git source control files it becomes
pointless to handle the tarballs of specific git revisions so drop this
part of the fetcher.

(From Poky rev: affcfe468a0c2993c0c4ace446248f85aee4315f)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2/git: Add nocheckout option to disable checkouts and always add scm...
Richard Purdie [Wed, 2 Feb 2011 12:48:32 +0000 (12:48 +0000)]
bitbake/fetch2/git: Add nocheckout option to disable checkouts and always add scm data to checkouts

(From Poky rev: 05048e0f9168ceb7d66f6ac79dcae42f1e696826)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: Add git unpack
Yu Ke [Tue, 18 Jan 2011 16:22:13 +0000 (00:22 +0800)]
bb.fetch2: Add git unpack

The git download method clones the git repository to the local machine. The unpack process
can be optimised to be a local to local machine clone or a direct readtree operation to the
destination using git.will clone git repo to local, so git unpack can be simplified
to only checkouting the code to the work dir. For fullclone case, we also
need to manually copy all the ref info, which is needed by the later do_kernel_checkout().
Rather than use hardlinks, we reference the repository using alternatives since the
download directory may be on a different filesystem.

[Change to use -s by Richard Purdie]

(From Poky rev: 3a3fc6b071f09970e7f79fb4f1a8f0013917371e)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Ensure the local revision counter takes a default value of 0, not...
Richard Purdie [Tue, 1 Feb 2011 23:33:19 +0000 (23:33 +0000)]
bitbake/fetch2: Ensure the local revision counter takes a default value of 0, not None

(From Poky rev: 05954ef4d7d882f34e2bb1f3bc44ad6f80c11d4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/fetch2: Instrument fetchers when making network access
Yu Ke [Mon, 24 Jan 2011 07:56:54 +0000 (15:56 +0800)]
bitbake/fetch2: Instrument fetchers when making network access

(From Poky rev: 029f8584d547c0792ffbe2f83451dcdfe2a3db7f)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: add "BB_NO_NETWORK" option
Yu Ke [Mon, 24 Jan 2011 06:58:26 +0000 (14:58 +0800)]
bb.fetch2: add "BB_NO_NETWORK" option

Sometime user want a purely local fetching, i.e. using local mirror without
any remote netowrk access. BB_NO_NETWORK option is introduced for this purpose

check_network_access() is the guard for BB_NO_NETWOKR option. it should be
put in any place that fetcher use network access

(From Poky rev: 098e8ded339f3bf864f3bad9871028176f70b12b)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: remove the obsolate Fetch.try_mirrors referrence
Yu Ke [Tue, 18 Jan 2011 16:17:39 +0000 (00:17 +0800)]
bb.fetch2: remove the obsolate Fetch.try_mirrors referrence

Fetch.try_mirrors is no longer exists, so the code is obsolate

(From Poky rev: 733de7596c2ed78a846541b3290f01c21ff96606)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agogit.py: split download to download() and build_mirror_data()
Yu Ke [Tue, 18 Jan 2011 15:35:30 +0000 (23:35 +0800)]
git.py: split download to download() and build_mirror_data()

the download is to fetch the source from URL, the build_mirror_data is
to create the mirror tar ball. the original go() method mix them together,
it is more clean to split them.

(From Poky rev: ef918a72de97fd2189e1c8487e371d13e18088fd)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: rename "go" with "download" to better reflect its functionality
Yu Ke [Tue, 18 Jan 2011 15:03:53 +0000 (23:03 +0800)]
bb.fetch2: rename "go" with "download" to better reflect its functionality

no functional change

(From Poky rev: e05918937c515dff845fcb4c9e94f8ecbea8c957)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2
Yu Ke [Tue, 18 Jan 2011 14:41:23 +0000 (22:41 +0800)]
bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2

there is case that we need to distingush bb.fetch and bb.fetch2,
and use different API for bb.fetch and bb.fetch2. so it is necessary
to add version info for distinguish purpose

(From Poky rev: f12e71484593039cd58b6e7fadd038b28b05d849)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: revise the Fetch.unpack API
Yu Ke [Tue, 18 Jan 2011 14:08:09 +0000 (22:08 +0800)]
bb.fetch2: revise the Fetch.unpack API

change the unpack to use the urldata and rootdir parameter
- urldata is the FetchData instance
- rootdir is the dir to put the extracted source. the original unpack
  use current dir (os.getcwd) as destination dir, which is not flexible
  and error-prone (error will occur if caller not chdir to dest dir)

(From Poky rev: 84ff79413a3b56f287f42b8ca1dd2ab194337c42)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: add unpack method in fetcher
Yu Ke [Tue, 18 Jan 2011 13:53:36 +0000 (21:53 +0800)]
bb.fetch2: add unpack method in fetcher

copy exactly the base.bbclass:oe_unpack_file() to bb.fetch2 as the code base

(From Poky rev: d8698b92ffcc2cbd29b57d3dc8e851d3d6c137f2)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoFetcher: only set __BB_DONT_CACHE when SRCREV = "${AUTOREV}"
Yu Ke [Mon, 27 Dec 2010 08:20:34 +0000 (16:20 +0800)]
Fetcher: only set __BB_DONT_CACHE when SRCREV = "${AUTOREV}"

we should cache SRCREV whenever possible, the only exception is
when SREREV is auto rev. so change the logic to only set __BB_DONT_CACHE
at SRCREV = "${AUTOREV}" case

(From Poky rev: 2b2dbf76126650750cb8a6778b837af6fe4c8b21)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoFetcher: break the "SRCREVINACTION" deadlock
Yu Ke [Mon, 27 Dec 2010 01:31:38 +0000 (09:31 +0800)]
Fetcher: break the "SRCREVINACTION" deadlock

Current fetcher has annoying "SRCREVINACTION" deadlock,
which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev():
get_srcrev()->setup_localpath()->srcrev_internal_helper()
->evaluate SRCREV->get_srcrev()

current fetcher resolve the deadlock by introducing a
"SRCREVINACTION" condition check. Althoguh it works, it is
indeed not clean.

This patch use antoehr idea to break the deadlock: break
the dependency among SRCREV and get_srcrev(), i.e. assign
a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet
this keyword, it will check and set the latest revision to
urldata.revision. get_srcrev later can use the urldata.revision
for value evaluation(SRCPV etc). In this case, SRCREV no longer
depends on get_srcrev, and there is not deadlock anymore.

(From Poky rev: 1589a1172f9432aed1cc9ce006f68cddf3073774)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2.bzr: add bzr urldata_init
Yu Ke [Mon, 27 Dec 2010 02:48:34 +0000 (10:48 +0800)]
bb.fetch2.bzr: add bzr urldata_init

move the bzr specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 550c3bd82115b4bdb8235da53cfc18b1dc39ad96)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2.hg: add hg urldata_init
Yu Ke [Mon, 27 Dec 2010 02:35:19 +0000 (10:35 +0800)]
bb.fetch2.hg: add hg urldata_init

move the hg specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: f684ff18a2b9565823a41750af369dee392f6142)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2.svn.py: add urldata_init
Yu Ke [Mon, 27 Dec 2010 02:07:04 +0000 (10:07 +0800)]
bb.fetch2.svn.py: add urldata_init

move the svn specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 7420e70f4f03a18f9d5a0d3226918088af1032bd)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2.git.py: add git urldata_init
Yu Ke [Mon, 27 Dec 2010 01:50:25 +0000 (09:50 +0800)]
bb.fetch2.git.py: add git urldata_init

move the git specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 54e34f6e255d1717beada23638a5783c9dda42ea)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobb.fetch2: add urldata_init call for Fetch class
Yu Ke [Sat, 8 Jan 2011 08:40:18 +0000 (16:40 +0800)]
bb.fetch2: add urldata_init call for Fetch class

FetchData has some fetch method specific data, and only fetch method knows how
to initialize it. originally it is mostly initialized in Fetch.localpath().
But now there is requirement to call Fetch.latest_revision() before
Fetch.localpath(), thus require another earlier place for initialization. so
urldata_init is introduced for this purpose. it will be called in FetchData:__init__
and make all the Fetch functions useable after that.

(From Poky rev: 2892cbfb114de1d2e2b5b479d5f928646ce5c831)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoBBHandler: remove bb.fetch referrence
Yu Ke [Sat, 8 Jan 2011 02:26:28 +0000 (10:26 +0800)]
BBHandler: remove bb.fetch referrence

BBHandler.py no longer use bb.fetch, so remove its import statement

(From Poky rev: 9e3a92c04a5be4932a0324af4a74bce871126e9d)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/utils.py: add glob name matching to remove
Saul Wold [Tue, 8 Feb 2011 17:24:12 +0000 (09:24 -0800)]
bitbake/utils.py: add glob name matching to remove

(From Poky rev: 07088f7711e0b53c4fac3685d15d406cf4793602)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agobitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level...
Richard Purdie [Mon, 7 Feb 2011 14:49:10 +0000 (14:49 +0000)]
bitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level values from formatter

(From Poky rev: 7586adb360d8075d3e97184dfcafb1b13ce5f838)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years ago__init__.py: Fix debug log level handling to correct debug output
Richard Purdie [Mon, 7 Feb 2011 14:46:22 +0000 (14:46 +0000)]
__init__.py: Fix debug log level handling to correct debug output

(From Poky rev: d7eebbe9dbf0d790d4af93466f5c27127cae0999)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoExport KRB5CCNAME variable
Javier Martin [Mon, 31 Jan 2011 11:27:54 +0000 (12:27 +0100)]
Export KRB5CCNAME variable

This allows fetching git repositories using Kerberos authentication.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years ago.gitignore: add *.pyo
Andreas Oberritter [Tue, 4 Jan 2011 15:32:07 +0000 (15:32 +0000)]
.gitignore: add *.pyo

* python generates pyo files if PYTHONOPTIMIZE is set

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoFix comparison with SRCREVINACTION constant
Javier Martin [Thu, 27 Jan 2011 08:38:21 +0000 (09:38 +0100)]
Fix comparison with SRCREVINACTION constant

Use '==' instead of 'is', otherwise it will always return
true since 'rev' and "SRCREVINACTION" are not the same object.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoevent: fix exception format
Chris Larson [Tue, 8 Feb 2011 22:35:03 +0000 (15:35 -0700)]
event: fix exception format

Forgot to concatenate the lines returned by format_exception.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoevent: limit the number of traceback entries to 5
Chris Larson [Tue, 8 Feb 2011 20:51:46 +0000 (13:51 -0700)]
event: limit the number of traceback entries to 5

This is slightly more sane.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agopersist_data: kill unreachable break line
Chris Larson [Tue, 8 Feb 2011 20:42:46 +0000 (13:42 -0700)]
persist_data: kill unreachable break line

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoShift traceback pre-formatting into LogHandler
Chris Larson [Tue, 8 Feb 2011 20:41:36 +0000 (13:41 -0700)]
Shift traceback pre-formatting into LogHandler

It's cleaner to leave it behaving as usual, passing the exception data in the
exc_info attribute of the LogRecord where it normally lies, and then let
LogHandler make it pickleable so it can be sent to the UI.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agopersist_data: loop on database lock for table creation
Chris Larson [Tue, 8 Feb 2011 17:41:58 +0000 (10:41 -0700)]
persist_data: loop on database lock for table creation

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agobuild: improve set -x handling
Chris Larson [Tue, 8 Feb 2011 16:53:47 +0000 (09:53 -0700)]
build: improve set -x handling

- set -x always, not just when debugging
- set -x immediately before executing the function, rather than at the top of
  the script, so the output is no longer cluttered with all the variable and
  function definitions

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoMerge branch '1.10'
Chris Larson [Mon, 7 Feb 2011 20:38:10 +0000 (13:38 -0700)]
Merge branch '1.10'

8 years agobitbake-layers: fix bug with env vars leaking in
Chris Larson [Mon, 7 Feb 2011 17:58:40 +0000 (10:58 -0700)]
bitbake-layers: fix bug with env vars leaking in

I forgot that bin/bitbake is what does the environment filtering based upon
BB_ENV_WHITELIST, etc.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agoAdd initial bitbake-layers script
Chris Larson [Mon, 7 Feb 2011 17:50:27 +0000 (10:50 -0700)]
Add initial bitbake-layers script

This script has subcommands which operate against your bitbake layers, either
displaying useful information, or acting against them.  Currently, it only
provides a show_appends command, which shows you what bbappends are in effect,
and warns you if you have appends which are not being utilized.

Currently, a bug exists when using this due to the DataContext stuff, but I'm
not certain as to the root cause, it appears to be the bb package relying
implicitly on the way the bitbake script does things.  A fix for that issue
will be forthcoming, as will further subcommands.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agobitbake/runqueue.py: Avoid starvation of events to the server
Richard Purdie [Fri, 28 Jan 2011 10:21:41 +0000 (10:21 +0000)]
bitbake/runqueue.py: Avoid starvation of events to the server

The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since
most new LogRecord events are larger than this it leads to a build up of data
which is only processed slowly, leading to a bottleneck and a slow down of
all bitbake processes.

Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging
this. A large value has been left in for the read() command just to ensure some
fairness amongst process handling if a task tries to log truly huge amounts of
data to the server, or goes crazy and ensures the main loop doesn't stall.

(From Poky rev: 06c6db7929c75f576a395fb442abe447b833fc3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
8 years agoAdd a few items to .gitignore
Chris Larson [Fri, 21 Jan 2011 01:51:52 +0000 (18:51 -0700)]
Add a few items to .gitignore

Signed-off-by: Chris Larson <chris_larson@mentor.com>
8 years agocache: fix docstring usage
Chris Larson [Thu, 20 Jan 2011 21:32:16 +0000 (14:32 -0700)]
cache: fix docstring usage

Use comments instead of docstrings where appropriate in CacheData.

Signed-off-by: Chris Larson <chris_larson@mentor.com>