bitbake.git
9 years agobitbake/fetch2/git: Fix broken variable reference
Richard Purdie [Tue, 8 Feb 2011 09:33:52 +0000 (09:33 +0000)]
bitbake/fetch2/git: Fix broken variable reference

(From Poky rev: a235c67479846590eb2c7236bec3360b23c3e9b6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Ensure unresolved branches are translated into revisions
Richard Purdie [Tue, 8 Feb 2011 09:27:59 +0000 (09:27 +0000)]
bitbake/fetch2/git: Ensure unresolved branches are translated into revisions

(From Poky rev: 5920e85c561624e657c126df58f5c378a8950bbc)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2: add try/finally to ensure lockfile is unlocked on failure
Saul Wold [Tue, 8 Feb 2011 01:29:46 +0000 (17:29 -0800)]
fetch2: add try/finally to ensure lockfile is unlocked on failure

(From Poky rev: 1544aa8ab4a80d529a001e27b473645f2caec87c)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2: add runfetchcmd to import for fetchers
Saul Wold [Mon, 7 Feb 2011 23:42:45 +0000 (15:42 -0800)]
fetch2: add runfetchcmd to import for fetchers

(From Poky rev: 232b6f3c92928c333ad1201aa8eb3706e7251cdf)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2:Fetch Make using the fn based cache optional
Richard Purdie [Mon, 7 Feb 2011 21:12:51 +0000 (21:12 +0000)]
bitbake/fetch2:Fetch Make using the fn based cache optional

(From Poky rev: 500c66337c7cb5e3044a02ef761097713e47f523)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Fix pickling issues with fetcher exceptions
Richard Purdie [Mon, 7 Feb 2011 20:46:42 +0000 (20:46 +0000)]
bitbake/fetch2: Fix pickling issues with fetcher exceptions

See the problems in http://bugs.python.org/issue1692335, need to set self.args
correctly.

(From Poky rev: d4b4b48addfa781d7b94965e0477974c3fb6dbb3)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs
Richard Purdie [Mon, 7 Feb 2011 15:28:05 +0000 (15:28 +0000)]
bitbake/fetch2: Update mirror processing to ensure we look for mirror tarballs

(From Poky rev: 94faffdaf6c13ce59987aab28383d66a9a0bf100)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug...
Richard Purdie [Mon, 7 Feb 2011 12:08:32 +0000 (12:08 +0000)]
bitbake/fetch2: Update forcefetch and mirror handling to clean up, simplfy and bug fix the code

(From Poky rev: 37624b97450f2ba3d6fad3e1e51818486451447e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Match Fetcher log domain to that in bb.msg
Richard Purdie [Mon, 7 Feb 2011 14:46:53 +0000 (14:46 +0000)]
bitbake/fetch2: Match Fetcher log domain to that in bb.msg

(From Poky rev: ded83ffc3bd86553c9b540795660985905c925ba)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may...
Richard Purdie [Mon, 7 Feb 2011 10:13:29 +0000 (10:13 +0000)]
bitbake/fetch2/git: use clonedir as ud.localfile too since the mirror tarball may not exist

(From Poky rev: 681bcf4e6b606dde2029d143805023a927285917)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for...
Richard Purdie [Mon, 7 Feb 2011 09:34:24 +0000 (09:34 +0000)]
bitbake/fetch2/git.py: Ensure that forcefetch operates in the correct directory for calling _contains_ref()

(From Poky rev: 330886826770ff6ec1449dc375cb4c3604b2736b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer alway...
Richard Purdie [Mon, 7 Feb 2011 09:13:44 +0000 (09:13 +0000)]
bitbake/fetch2/git: Fix localpath to point at the clone repo since we no longer always generate a mirror tarball and it isn't a good guide to fetcher success

(From Poky rev: 917d3e9697acefe308e7139e86df37a072ee3500)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Use True instead of integer values
Richard Purdie [Fri, 4 Feb 2011 14:40:41 +0000 (14:40 +0000)]
bitbake/fetch2: Use True instead of integer values

(From Poky rev: 7202a77134029cb37540c785ce0161a4dd574853)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Drop name/override ordering backwards compatibility code
Richard Purdie [Fri, 4 Feb 2011 14:37:09 +0000 (14:37 +0000)]
bitbake/fetch2: Drop name/override ordering backwards compatibility code

(From Poky rev: 8f2abf4a9f000d42e98c4936e393bd5033a1af48)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/svn: The concept of using dates with svn is totally insane, drop support
Richard Purdie [Fri, 4 Feb 2011 13:46:31 +0000 (13:46 +0000)]
bitbake/fetch2/svn: The concept of using dates with svn is totally insane, drop support

(From Poky rev: ee6f34483a9bfeca5b5c3656fd62bb4849d5fbb8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class
Richard Purdie [Fri, 4 Feb 2011 13:14:03 +0000 (13:14 +0000)]
bitbake/fetch2: Define a new interface to the fetcher code though the 'Fetch' class

(From Poky rev: ca7adf75295c2a6041b891bfa61e0b4bc2f7c860)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Fix typo
Richard Purdie [Fri, 4 Feb 2011 11:06:16 +0000 (11:06 +0000)]
bitbake/fetch2: Fix typo

(From Poky rev: 08a9fef4fa0036708f29bd150d7ecf66354e5475)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Drop legacy CVSDATE support
Richard Purdie [Fri, 4 Feb 2011 11:03:34 +0000 (11:03 +0000)]
bitbake/fetch2: Drop legacy CVSDATE support

(From Poky rev: 806d1c25f8ca8ab5116652e6f82f7d9ee5774278)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate
Richard Purdie [Fri, 4 Feb 2011 11:02:15 +0000 (11:02 +0000)]
bitbake/fetch2: Move getSRCDate to FetchData class where is more appropriate

(From Poky rev: f60d9d6bd4bb7691b62bf1f4b1fbeaa1faa53879)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in...
Richard Purdie [Fri, 4 Feb 2011 10:59:42 +0000 (10:59 +0000)]
bitbake/fetch2: Make srcrev_internal_helper a normal function, doesn't belong in the FetchMethod class

(From Poky rev: 831c6c1d3c23c50f9c55a6bbf8dc6311bca3881b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Rename Fetch class to FetchMethod
Richard Purdie [Fri, 4 Feb 2011 10:49:27 +0000 (10:49 +0000)]
bitbake/fetch2: Rename Fetch class to FetchMethod

(From Poky rev: ab0dd1397491478ee6149283e5ba8775dd8cdc3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Drop old md5 handling code
Richard Purdie [Fri, 4 Feb 2011 10:30:54 +0000 (10:30 +0000)]
bitbake/fetch2: Drop old md5 handling code

Drop some old md5 functions since we have improved functionality now which includes
sha256 checksum support. This stops each download being md5 checksumed twice.

Also change ".md5" stamp extentions to ".done" to better describe its use as a
download complete marker file and no longer write the md5 sum to the files.

(From Poky rev: 74b71864fed79ce60e721945c8e239b3ebf49200)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Rewrite and improve exception handling, reusing core functions for...
Richard Purdie [Fri, 4 Feb 2011 10:26:21 +0000 (10:26 +0000)]
bitbake/fetch2: Rewrite and improve exception handling, reusing core functions for common operations where possible

(From Poky rev: d08397ba4d1331993300eacbb2f78fcfef19c1cf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException...
Richard Purdie [Thu, 3 Feb 2011 23:23:11 +0000 (23:23 +0000)]
bitbake/fetch2: Have all fetcher exceptions derived from a common BBFetchException class

(From Poky rev: f6eefb3ca3bb2a5ea0ec1364bdb0bc41ae58c815)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Drop unused parameter for localpath() and update comments
Richard Purdie [Thu, 3 Feb 2011 22:02:09 +0000 (22:02 +0000)]
bitbake/fetch2: Drop unused parameter for localpath() and update comments

(From Poky rev: 8daab5b95157dda6854fe6bf1929f911fe3cf25e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Simplfy setup_srcrevs
Richard Purdie [Thu, 3 Feb 2011 21:43:01 +0000 (21:43 +0000)]
bitbake/fetch2: Simplfy setup_srcrevs

(From Poky rev: 7e4fbfc1c1887a1a0507b60244aa53b8b1994edd)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Simplify localpath variable handling FetchData init
Richard Purdie [Thu, 3 Feb 2011 21:40:03 +0000 (21:40 +0000)]
bitbake/fetch2: Simplify localpath variable handling FetchData init

(From Poky rev: 49a022d25d35115e7286e2ca2530566da2d71aa8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Define a sane localpath function and remove code duplication
Richard Purdie [Thu, 3 Feb 2011 21:20:39 +0000 (21:20 +0000)]
bitbake/fetch2: Define a sane localpath function and remove code duplication

(From Poky rev: 5a52cb80d6d62c704692ad6cfa93426c4a17bc52)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Move ud.localfile setup into urldata_init
Richard Purdie [Thu, 3 Feb 2011 21:17:26 +0000 (21:17 +0000)]
bitbake/fetch2: Move ud.localfile setup into urldata_init

(From Poky rev: 972eb5fababb33b5537fcfbbaf8e33ea820f0fee)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Drop horrible SRCREV hack which is no longer needed
Richard Purdie [Thu, 3 Feb 2011 20:33:07 +0000 (20:33 +0000)]
bitbake/fetch2: Drop horrible SRCREV hack which is no longer needed

(From Poky rev: 0e5404cedadaea6738ef6468d7eb8a24d23d9bab)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Switch to completely use bare clones
Richard Purdie [Sun, 6 Feb 2011 10:20:18 +0000 (10:20 +0000)]
bitbake/fetch2/git: Switch to completely use bare clones

We no longer need index/workdir support in the mirror tree and it causes all
kind of reference naming problems.Simplifying the code to remove this and use
just bare clones addresses this problem.

We increase the "version" number on the mirror tarballs to reflect the change
and ensure older mirror tarballs are not used as they would break.

(From Poky rev: e4c1e99f234491af1c0fc886e73deedaef108a1a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2/git.py: Fix problem when fetching with no origin
Saul Wold [Sat, 5 Feb 2011 22:11:25 +0000 (14:11 -0800)]
fetch2/git.py: Fix problem when fetching with no origin

When updating a git repo, its possible to not have an origin,
so fail gracefully with a debug message.

(From Poky rev: 192f631129e25b959d501c46c3156b7037b49c22)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Ensure we prune origin to remove stale branches
Richard Purdie [Sat, 5 Feb 2011 13:16:45 +0000 (14:16 +0100)]
bitbake/fetch2/git: Ensure we prune origin to remove stale branches

Thanks to Bruce Ashfield for the tip.

(From Poky rev: 96810cade04cad44d70a7c6e51aacdf89caef92b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags...
Richard Purdie [Thu, 3 Feb 2011 20:16:05 +0000 (20:16 +0000)]
bitbake/fetch2/git: Ensure deleted branches upstream are removed, all branch tags are forced to the upstream revisions and all revisions and tags are always fetched

(From Poky rev: dfc5faf717ac1f917af398097d5295128548a55d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Remove old local file acceleration code
Richard Purdie [Thu, 3 Feb 2011 18:24:25 +0000 (18:24 +0000)]
bitbake/fetch2: Remove old local file acceleration code

Since local mirror fetches are always symlinked from the download directory
directly, there is no need for this premirrors hack which doesn't cover
mirrors and also abuses the localpath variable with inconsistent results.

(From Poky rev: 282a828f3dc373d8f1397827ebbe1be1c54f2d2a)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory
Richard Purdie [Thu, 3 Feb 2011 18:22:06 +0000 (18:22 +0000)]
bitbake/fetch2: Ensure that mirror fetches are symlinked from the download directory

When files are fetched from a mirror source that happens to be local,
ensure links are created for the file since subsequent fetch calls
can then follow the links to find files.

Any other approach such as the existing manipulations of localpath
internally to the fetcher are prone to errors, races and other issues.

(From Poky rev: fa9fda05e1d269446b51050195b891346482e8bb)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: When encoding a file:// url, drop user and host information
Richard Purdie [Thu, 3 Feb 2011 18:19:23 +0000 (18:19 +0000)]
bitbake/fetch2: When encoding a file:// url, drop user and host information

When processing a cvs SRC_URI to a file:// mirror, the user and host information
will break the mirror processing. This patch addresses it by only constructing
valid urls.

(From Poky rev: 7f99605562119a13a2510a3c990e3cf577ad764e)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/lib/bb/fetch2: Assign a default name in the names array as empty strings...
Richard Purdie [Thu, 3 Feb 2011 00:16:23 +0000 (00:16 +0000)]
bitbake/lib/bb/fetch2: Assign a default name in the names array as empty strings as names cause problems for python

(From Poky rev: a55d86b4051f6d063f9a67db026f7da6f9b50cc6)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Add missing parameter to localcount_internal_helper
Richard Purdie [Wed, 2 Feb 2011 18:48:13 +0000 (18:48 +0000)]
bitbake/fetch2: Add missing parameter to localcount_internal_helper

(From Poky rev: 08cd6c1cb9639e958e0c60a02e317a22cf43f880)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Ensure target directory exists when copying files
Richard Purdie [Wed, 2 Feb 2011 17:46:37 +0000 (17:46 +0000)]
bitbake/fetch2/git: Ensure target directory exists when copying files

(From Poky rev: adfa6c40dadebb18bfd457859300d8c093b007f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2: Allow multiple src rev to be used in one src_uri
Yu Ke [Wed, 26 Jan 2011 12:14:06 +0000 (20:14 +0800)]
fetch2: Allow multiple src rev to be used in one src_uri

* SRC_URI format:
the SRC_URI are extended to allow multiple src rev:
name=<name1>,<name2>,...<name-n>
branch=<branch1>,<branch2>,...,<branch-n>
also SRCREV can be defined with
SRCREV_<name1> = xxxxx
SRCREV_<name2> = xxxxx

* FetchData extention
to support multiple src rev, several FetchData data are added:
- FetchData.names: list of name in SRC_URI,  one name per srcrev. name is the index of revision and branch
- FetchData.revisions: dictionary of name->revision.
- FetchData.branches: dictionary of name->branch.

For example, linux-yocto recipes becomes:
SRC_URI = "git://git.pokylinux.org/linux-yocto-2.6.37;protocol=git;branch=${KBRANCH},meta;name=machine,meta"

FetchData.names = ['machine', 'meta']
FetchData.revisions = { 'machine':xxxxx, 'meta':xxxxxx }
FetchData.branches = { 'machine':${KBRANCH}, 'meta':'meta'}

* generic revision handling extension
the related revision handling code in fetch2.__init__.py are changed accordingly. the major change is add name parameter to indicate which src rev to handling. originally there is one src rev per FetchData, so FetchData parameter is enough. now since one FetchData has multiple src rev, it is necessary to use FetchData + name to specifiy src rev.

* git extension
git fetcher are also revised to take advantage of the multiple src rev in FetchData. especially the download() method are enhanced to fetch multiple src rev.

* other fetcher (svn, hg, ...) does not support multiple src rev. they just sync the API to add name, and then simply ignore the name. no actually functional change

(From Poky rev: 0e837e6844be449659bb96a1498e54a8b9442d13)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agogit.py: Remove the source tree tarball archives
Yu Ke [Tue, 18 Jan 2011 16:58:42 +0000 (00:58 +0800)]
git.py: Remove the source tree tarball archives

Since we're now always providing the git source control files it becomes
pointless to handle the tarballs of specific git revisions so drop this
part of the fetcher.

(From Poky rev: affcfe468a0c2993c0c4ace446248f85aee4315f)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2/git: Add nocheckout option to disable checkouts and always add scm...
Richard Purdie [Wed, 2 Feb 2011 12:48:32 +0000 (12:48 +0000)]
bitbake/fetch2/git: Add nocheckout option to disable checkouts and always add scm data to checkouts

(From Poky rev: 05048e0f9168ceb7d66f6ac79dcae42f1e696826)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: Add git unpack
Yu Ke [Tue, 18 Jan 2011 16:22:13 +0000 (00:22 +0800)]
bb.fetch2: Add git unpack

The git download method clones the git repository to the local machine. The unpack process
can be optimised to be a local to local machine clone or a direct readtree operation to the
destination using git.will clone git repo to local, so git unpack can be simplified
to only checkouting the code to the work dir. For fullclone case, we also
need to manually copy all the ref info, which is needed by the later do_kernel_checkout().
Rather than use hardlinks, we reference the repository using alternatives since the
download directory may be on a different filesystem.

[Change to use -s by Richard Purdie]

(From Poky rev: 3a3fc6b071f09970e7f79fb4f1a8f0013917371e)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Ensure the local revision counter takes a default value of 0, not...
Richard Purdie [Tue, 1 Feb 2011 23:33:19 +0000 (23:33 +0000)]
bitbake/fetch2: Ensure the local revision counter takes a default value of 0, not None

(From Poky rev: 05954ef4d7d882f34e2bb1f3bc44ad6f80c11d4c)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/fetch2: Instrument fetchers when making network access
Yu Ke [Mon, 24 Jan 2011 07:56:54 +0000 (15:56 +0800)]
bitbake/fetch2: Instrument fetchers when making network access

(From Poky rev: 029f8584d547c0792ffbe2f83451dcdfe2a3db7f)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: add "BB_NO_NETWORK" option
Yu Ke [Mon, 24 Jan 2011 06:58:26 +0000 (14:58 +0800)]
bb.fetch2: add "BB_NO_NETWORK" option

Sometime user want a purely local fetching, i.e. using local mirror without
any remote netowrk access. BB_NO_NETWORK option is introduced for this purpose

check_network_access() is the guard for BB_NO_NETWOKR option. it should be
put in any place that fetcher use network access

(From Poky rev: 098e8ded339f3bf864f3bad9871028176f70b12b)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: remove the obsolate Fetch.try_mirrors referrence
Yu Ke [Tue, 18 Jan 2011 16:17:39 +0000 (00:17 +0800)]
bb.fetch2: remove the obsolate Fetch.try_mirrors referrence

Fetch.try_mirrors is no longer exists, so the code is obsolate

(From Poky rev: 733de7596c2ed78a846541b3290f01c21ff96606)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agogit.py: split download to download() and build_mirror_data()
Yu Ke [Tue, 18 Jan 2011 15:35:30 +0000 (23:35 +0800)]
git.py: split download to download() and build_mirror_data()

the download is to fetch the source from URL, the build_mirror_data is
to create the mirror tar ball. the original go() method mix them together,
it is more clean to split them.

(From Poky rev: ef918a72de97fd2189e1c8487e371d13e18088fd)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: rename "go" with "download" to better reflect its functionality
Yu Ke [Tue, 18 Jan 2011 15:03:53 +0000 (23:03 +0800)]
bb.fetch2: rename "go" with "download" to better reflect its functionality

no functional change

(From Poky rev: e05918937c515dff845fcb4c9e94f8ecbea8c957)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2
Yu Ke [Tue, 18 Jan 2011 14:41:23 +0000 (22:41 +0800)]
bb.fetch: add fetch version to distinguish bb.fetch and bb.fetch2

there is case that we need to distingush bb.fetch and bb.fetch2,
and use different API for bb.fetch and bb.fetch2. so it is necessary
to add version info for distinguish purpose

(From Poky rev: f12e71484593039cd58b6e7fadd038b28b05d849)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: revise the Fetch.unpack API
Yu Ke [Tue, 18 Jan 2011 14:08:09 +0000 (22:08 +0800)]
bb.fetch2: revise the Fetch.unpack API

change the unpack to use the urldata and rootdir parameter
- urldata is the FetchData instance
- rootdir is the dir to put the extracted source. the original unpack
  use current dir (os.getcwd) as destination dir, which is not flexible
  and error-prone (error will occur if caller not chdir to dest dir)

(From Poky rev: 84ff79413a3b56f287f42b8ca1dd2ab194337c42)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: add unpack method in fetcher
Yu Ke [Tue, 18 Jan 2011 13:53:36 +0000 (21:53 +0800)]
bb.fetch2: add unpack method in fetcher

copy exactly the base.bbclass:oe_unpack_file() to bb.fetch2 as the code base

(From Poky rev: d8698b92ffcc2cbd29b57d3dc8e851d3d6c137f2)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoFetcher: only set __BB_DONT_CACHE when SRCREV = "${AUTOREV}"
Yu Ke [Mon, 27 Dec 2010 08:20:34 +0000 (16:20 +0800)]
Fetcher: only set __BB_DONT_CACHE when SRCREV = "${AUTOREV}"

we should cache SRCREV whenever possible, the only exception is
when SREREV is auto rev. so change the logic to only set __BB_DONT_CACHE
at SRCREV = "${AUTOREV}" case

(From Poky rev: 2b2dbf76126650750cb8a6778b837af6fe4c8b21)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoFetcher: break the "SRCREVINACTION" deadlock
Yu Ke [Mon, 27 Dec 2010 01:31:38 +0000 (09:31 +0800)]
Fetcher: break the "SRCREVINACTION" deadlock

Current fetcher has annoying "SRCREVINACTION" deadlock,
which occurs when SRCREV=${AUTOREV}=@bb.fetch.get_srcrev():
get_srcrev()->setup_localpath()->srcrev_internal_helper()
->evaluate SRCREV->get_srcrev()

current fetcher resolve the deadlock by introducing a
"SRCREVINACTION" condition check. Althoguh it works, it is
indeed not clean.

This patch use antoehr idea to break the deadlock: break
the dependency among SRCREV and get_srcrev(), i.e. assign
a specific keyword "AUTOINC" to AUTOREV. when Fetcher meet
this keyword, it will check and set the latest revision to
urldata.revision. get_srcrev later can use the urldata.revision
for value evaluation(SRCPV etc). In this case, SRCREV no longer
depends on get_srcrev, and there is not deadlock anymore.

(From Poky rev: 1589a1172f9432aed1cc9ce006f68cddf3073774)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2.bzr: add bzr urldata_init
Yu Ke [Mon, 27 Dec 2010 02:48:34 +0000 (10:48 +0800)]
bb.fetch2.bzr: add bzr urldata_init

move the bzr specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 550c3bd82115b4bdb8235da53cfc18b1dc39ad96)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2.hg: add hg urldata_init
Yu Ke [Mon, 27 Dec 2010 02:35:19 +0000 (10:35 +0800)]
bb.fetch2.hg: add hg urldata_init

move the hg specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: f684ff18a2b9565823a41750af369dee392f6142)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2.svn.py: add urldata_init
Yu Ke [Mon, 27 Dec 2010 02:07:04 +0000 (10:07 +0800)]
bb.fetch2.svn.py: add urldata_init

move the svn specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 7420e70f4f03a18f9d5a0d3226918088af1032bd)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2.git.py: add git urldata_init
Yu Ke [Mon, 27 Dec 2010 01:50:25 +0000 (09:50 +0800)]
bb.fetch2.git.py: add git urldata_init

move the git specific urldata init from localpath to urldata_init
so that it can be called early

(From Poky rev: 54e34f6e255d1717beada23638a5783c9dda42ea)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: add urldata_init call for Fetch class
Yu Ke [Sat, 8 Jan 2011 08:40:18 +0000 (16:40 +0800)]
bb.fetch2: add urldata_init call for Fetch class

FetchData has some fetch method specific data, and only fetch method knows how
to initialize it. originally it is mostly initialized in Fetch.localpath().
But now there is requirement to call Fetch.latest_revision() before
Fetch.localpath(), thus require another earlier place for initialization. so
urldata_init is introduced for this purpose. it will be called in FetchData:__init__
and make all the Fetch functions useable after that.

(From Poky rev: 2892cbfb114de1d2e2b5b479d5f928646ce5c831)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoBBHandler: remove bb.fetch referrence
Yu Ke [Sat, 8 Jan 2011 02:26:28 +0000 (10:26 +0800)]
BBHandler: remove bb.fetch referrence

BBHandler.py no longer use bb.fetch, so remove its import statement

(From Poky rev: 9e3a92c04a5be4932a0324af4a74bce871126e9d)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/utils.py: add glob name matching to remove
Saul Wold [Tue, 8 Feb 2011 17:24:12 +0000 (09:24 -0800)]
bitbake/utils.py: add glob name matching to remove

(From Poky rev: 07088f7711e0b53c4fac3685d15d406cf4793602)

Signed-off-by: Saul Wold <sgw@linux.intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level...
Richard Purdie [Mon, 7 Feb 2011 14:49:10 +0000 (14:49 +0000)]
bitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level values from formatter

(From Poky rev: 7586adb360d8075d3e97184dfcafb1b13ce5f838)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years ago__init__.py: Fix debug log level handling to correct debug output
Richard Purdie [Mon, 7 Feb 2011 14:46:22 +0000 (14:46 +0000)]
__init__.py: Fix debug log level handling to correct debug output

(From Poky rev: d7eebbe9dbf0d790d4af93466f5c27127cae0999)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoExport KRB5CCNAME variable
Javier Martin [Mon, 31 Jan 2011 11:27:54 +0000 (12:27 +0100)]
Export KRB5CCNAME variable

This allows fetching git repositories using Kerberos authentication.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years ago.gitignore: add *.pyo
Andreas Oberritter [Tue, 4 Jan 2011 15:32:07 +0000 (15:32 +0000)]
.gitignore: add *.pyo

* python generates pyo files if PYTHONOPTIMIZE is set

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoFix comparison with SRCREVINACTION constant
Javier Martin [Thu, 27 Jan 2011 08:38:21 +0000 (09:38 +0100)]
Fix comparison with SRCREVINACTION constant

Use '==' instead of 'is', otherwise it will always return
true since 'rev' and "SRCREVINACTION" are not the same object.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoevent: fix exception format
Chris Larson [Tue, 8 Feb 2011 22:35:03 +0000 (15:35 -0700)]
event: fix exception format

Forgot to concatenate the lines returned by format_exception.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoevent: limit the number of traceback entries to 5
Chris Larson [Tue, 8 Feb 2011 20:51:46 +0000 (13:51 -0700)]
event: limit the number of traceback entries to 5

This is slightly more sane.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agopersist_data: kill unreachable break line
Chris Larson [Tue, 8 Feb 2011 20:42:46 +0000 (13:42 -0700)]
persist_data: kill unreachable break line

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoShift traceback pre-formatting into LogHandler
Chris Larson [Tue, 8 Feb 2011 20:41:36 +0000 (13:41 -0700)]
Shift traceback pre-formatting into LogHandler

It's cleaner to leave it behaving as usual, passing the exception data in the
exc_info attribute of the LogRecord where it normally lies, and then let
LogHandler make it pickleable so it can be sent to the UI.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agopersist_data: loop on database lock for table creation
Chris Larson [Tue, 8 Feb 2011 17:41:58 +0000 (10:41 -0700)]
persist_data: loop on database lock for table creation

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobuild: improve set -x handling
Chris Larson [Tue, 8 Feb 2011 16:53:47 +0000 (09:53 -0700)]
build: improve set -x handling

- set -x always, not just when debugging
- set -x immediately before executing the function, rather than at the top of
  the script, so the output is no longer cluttered with all the variable and
  function definitions

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoMerge branch '1.10'
Chris Larson [Mon, 7 Feb 2011 20:38:10 +0000 (13:38 -0700)]
Merge branch '1.10'

9 years agobitbake-layers: fix bug with env vars leaking in
Chris Larson [Mon, 7 Feb 2011 17:58:40 +0000 (10:58 -0700)]
bitbake-layers: fix bug with env vars leaking in

I forgot that bin/bitbake is what does the environment filtering based upon
BB_ENV_WHITELIST, etc.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoAdd initial bitbake-layers script
Chris Larson [Mon, 7 Feb 2011 17:50:27 +0000 (10:50 -0700)]
Add initial bitbake-layers script

This script has subcommands which operate against your bitbake layers, either
displaying useful information, or acting against them.  Currently, it only
provides a show_appends command, which shows you what bbappends are in effect,
and warns you if you have appends which are not being utilized.

Currently, a bug exists when using this due to the DataContext stuff, but I'm
not certain as to the root cause, it appears to be the bb package relying
implicitly on the way the bitbake script does things.  A fix for that issue
will be forthcoming, as will further subcommands.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/runqueue.py: Avoid starvation of events to the server
Richard Purdie [Fri, 28 Jan 2011 10:21:41 +0000 (10:21 +0000)]
bitbake/runqueue.py: Avoid starvation of events to the server

The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since
most new LogRecord events are larger than this it leads to a build up of data
which is only processed slowly, leading to a bottleneck and a slow down of
all bitbake processes.

Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging
this. A large value has been left in for the read() command just to ensure some
fairness amongst process handling if a task tries to log truly huge amounts of
data to the server, or goes crazy and ensures the main loop doesn't stall.

(From Poky rev: 06c6db7929c75f576a395fb442abe447b833fc3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoAdd a few items to .gitignore
Chris Larson [Fri, 21 Jan 2011 01:51:52 +0000 (18:51 -0700)]
Add a few items to .gitignore

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agocache: fix docstring usage
Chris Larson [Thu, 20 Jan 2011 21:32:16 +0000 (14:32 -0700)]
cache: fix docstring usage

Use comments instead of docstrings where appropriate in CacheData.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agocooker: also respond to SystemExit for execute_runqueue
Chris Larson [Mon, 17 Jan 2011 15:50:40 +0000 (08:50 -0700)]
cooker: also respond to SystemExit for execute_runqueue

Without this, bitbake appears to hang when encountering an invalid task.  In
reality, the SystemExit flows up uncaught, the server may go away, and the UI
doesn't know about it, so requires us to ^C to exit.

This is a temporary measure.  Poky's bitbake handles invalid tasks
differently, and avoids this *particular* issue, but we do need to ensure that
the server responds to SystemExit from the idle function in a sane way.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/knotty.py: Ensure task note messages are only surpressed at default log levels
Richard Purdie [Wed, 12 Jan 2011 13:45:39 +0000 (13:45 +0000)]
bitbake/knotty.py: Ensure task note messages are only surpressed at default log levels

(From Poky rev: 3f08b7f1eadc8d181c76339d14c7abb4f49cad57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agocodeparser: fix spacing in diagnostic messages
Bernhard Reutner-Fischer [Tue, 11 Jan 2011 18:21:55 +0000 (19:21 +0100)]
codeparser: fix spacing in diagnostic messages

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years ago*: use utils.remove() some more
Bernhard Reutner-Fischer [Mon, 10 Jan 2011 22:05:19 +0000 (23:05 +0100)]
*: use utils.remove() some more

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years agobuild: remove duplicate import of utils
Bernhard Reutner-Fischer [Mon, 10 Jan 2011 15:32:56 +0000 (16:32 +0100)]
build: remove duplicate import of utils

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years agorunqueue.py: Call bb.cache.Cache.loadDataFull function directly when dumping signatur...
Richard Purdie [Mon, 10 Jan 2011 22:34:01 +0000 (22:34 +0000)]
runqueue.py: Call bb.cache.Cache.loadDataFull function directly when dumping signatures as per the rest of runqueue

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the BasicHash...
Richard Purdie [Mon, 10 Jan 2011 21:59:34 +0000 (21:59 +0000)]
bitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the BasicHash siggen code

(From Poky rev: 48e567bb477ad468d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agomsg: no need to hardcode the logging levels
Chris Larson [Mon, 10 Jan 2011 19:30:05 +0000 (12:30 -0700)]
msg: no need to hardcode the logging levels

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: properly highlight python inside of ${@}
Chris Larson [Mon, 10 Jan 2011 19:00:41 +0000 (12:00 -0700)]
vim: properly highlight python inside of ${@}

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: handle highlighting exports without assignment
Chris Larson [Mon, 10 Jan 2011 19:00:09 +0000 (12:00 -0700)]
vim: handle highlighting exports without assignment

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: add an ftplugin for indentation settings
Chris Larson [Mon, 10 Jan 2011 18:57:59 +0000 (11:57 -0700)]
vim: add an ftplugin for indentation settings

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake: handle IOError from event_queue.get
Chris Larson [Mon, 10 Jan 2011 18:54:52 +0000 (11:54 -0700)]
bitbake: handle IOError from event_queue.get

These can occur in a particular ^C shutdown, so just break out of the event
queue empty loop if we encounter it.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoInject taskpid into log records via our log handler
Chris Larson [Mon, 10 Jan 2011 16:20:50 +0000 (09:20 -0700)]
Inject taskpid into log records via our log handler

It turns out that while log filters added with addFilter are only associated
with that logger, and not its children, handlers are inherited, and handlers
can be filters.   So, let's add filtering to our existing LogHandler class
which dispatches our log records as bitbake events.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoSwitch to fetch2 fetcher code based on the environment variable BBFETCH2
Richard Purdie [Mon, 10 Jan 2011 18:12:53 +0000 (18:12 +0000)]
Switch to fetch2 fetcher code based on the environment variable BBFETCH2

(From Poky rev: af06e4a8efbcaff657c397c34f2cb893d2d7f7f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: replace bb.fetch with bb.fetch2 in the bb.fetch
Yu Ke [Mon, 10 Jan 2011 18:05:07 +0000 (18:05 +0000)]
bb.fetch2: replace bb.fetch with bb.fetch2 in the bb.fetch

bb.fetch2 is copied from bb.fetch, and has many bb.fetch referrence.
Fix these referrence with bb.fetch2 referrence

(From Poky rev: c64b445f38179a117d39319f762f9dc2435da0ea)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2/git: Remove duplicated code only added for backwards compatibility
Richard Purdie [Mon, 10 Jan 2011 14:26:56 +0000 (14:26 +0000)]
fetch2/git: Remove duplicated code only added for backwards compatibility

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake: copy bb.fetch to bb.fetch2 as initial code base for fetcher overhaul
Yu Ke [Mon, 10 Jan 2011 14:25:04 +0000 (14:25 +0000)]
bitbake: copy bb.fetch to bb.fetch2 as initial code base for fetcher overhaul

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoutils.py: Update log message to use the logger calls
Richard Purdie [Mon, 10 Jan 2011 13:19:29 +0000 (13:19 +0000)]
utils.py: Update log message to use the logger calls

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoevent.py: Improve debug about invalid events
Richard Purdie [Mon, 10 Jan 2011 13:08:09 +0000 (13:08 +0000)]
event.py: Improve debug about invalid events

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agopersist_data: Don't loop forever waiting on database locks
Richard Purdie [Mon, 10 Jan 2011 13:00:39 +0000 (13:00 +0000)]
persist_data: Don't loop forever waiting on database locks

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignored
Richard Purdie [Thu, 6 Jan 2011 19:48:47 +0000 (19:48 +0000)]
bitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignored

(From Poky rev: df8569b4d89ce83e3cafd87f2f37b795d1bfbd6d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>