bitbake.git
9 years agobitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level...
Richard Purdie [Mon, 7 Feb 2011 14:49:10 +0000 (14:49 +0000)]
bitbake/msg: Ensure lower level debug messages have DEBUG prefix and reuse log level values from formatter

(From Poky rev: 7586adb360d8075d3e97184dfcafb1b13ce5f838)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years ago__init__.py: Fix debug log level handling to correct debug output
Richard Purdie [Mon, 7 Feb 2011 14:46:22 +0000 (14:46 +0000)]
__init__.py: Fix debug log level handling to correct debug output

(From Poky rev: d7eebbe9dbf0d790d4af93466f5c27127cae0999)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoExport KRB5CCNAME variable
Javier Martin [Mon, 31 Jan 2011 11:27:54 +0000 (12:27 +0100)]
Export KRB5CCNAME variable

This allows fetching git repositories using Kerberos authentication.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years ago.gitignore: add *.pyo
Andreas Oberritter [Tue, 4 Jan 2011 15:32:07 +0000 (15:32 +0000)]
.gitignore: add *.pyo

* python generates pyo files if PYTHONOPTIMIZE is set

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoFix comparison with SRCREVINACTION constant
Javier Martin [Thu, 27 Jan 2011 08:38:21 +0000 (09:38 +0100)]
Fix comparison with SRCREVINACTION constant

Use '==' instead of 'is', otherwise it will always return
true since 'rev' and "SRCREVINACTION" are not the same object.

Signed-off-by: Javier Martin <javier.martin@vista-silicon.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoevent: fix exception format
Chris Larson [Tue, 8 Feb 2011 22:35:03 +0000 (15:35 -0700)]
event: fix exception format

Forgot to concatenate the lines returned by format_exception.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoevent: limit the number of traceback entries to 5
Chris Larson [Tue, 8 Feb 2011 20:51:46 +0000 (13:51 -0700)]
event: limit the number of traceback entries to 5

This is slightly more sane.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agopersist_data: kill unreachable break line
Chris Larson [Tue, 8 Feb 2011 20:42:46 +0000 (13:42 -0700)]
persist_data: kill unreachable break line

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoShift traceback pre-formatting into LogHandler
Chris Larson [Tue, 8 Feb 2011 20:41:36 +0000 (13:41 -0700)]
Shift traceback pre-formatting into LogHandler

It's cleaner to leave it behaving as usual, passing the exception data in the
exc_info attribute of the LogRecord where it normally lies, and then let
LogHandler make it pickleable so it can be sent to the UI.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agopersist_data: loop on database lock for table creation
Chris Larson [Tue, 8 Feb 2011 17:41:58 +0000 (10:41 -0700)]
persist_data: loop on database lock for table creation

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobuild: improve set -x handling
Chris Larson [Tue, 8 Feb 2011 16:53:47 +0000 (09:53 -0700)]
build: improve set -x handling

- set -x always, not just when debugging
- set -x immediately before executing the function, rather than at the top of
  the script, so the output is no longer cluttered with all the variable and
  function definitions

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoMerge branch '1.10'
Chris Larson [Mon, 7 Feb 2011 20:38:10 +0000 (13:38 -0700)]
Merge branch '1.10'

9 years agobitbake-layers: fix bug with env vars leaking in
Chris Larson [Mon, 7 Feb 2011 17:58:40 +0000 (10:58 -0700)]
bitbake-layers: fix bug with env vars leaking in

I forgot that bin/bitbake is what does the environment filtering based upon
BB_ENV_WHITELIST, etc.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoAdd initial bitbake-layers script
Chris Larson [Mon, 7 Feb 2011 17:50:27 +0000 (10:50 -0700)]
Add initial bitbake-layers script

This script has subcommands which operate against your bitbake layers, either
displaying useful information, or acting against them.  Currently, it only
provides a show_appends command, which shows you what bbappends are in effect,
and warns you if you have appends which are not being utilized.

Currently, a bug exists when using this due to the DataContext stuff, but I'm
not certain as to the root cause, it appears to be the bb package relying
implicitly on the way the bitbake script does things.  A fix for that issue
will be forthcoming, as will further subcommands.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/runqueue.py: Avoid starvation of events to the server
Richard Purdie [Fri, 28 Jan 2011 10:21:41 +0000 (10:21 +0000)]
bitbake/runqueue.py: Avoid starvation of events to the server

The server UI was reading 1024 bytes, then sleeping for 0.25 seconds. Since
most new LogRecord events are larger than this it leads to a build up of data
which is only processed slowly, leading to a bottleneck and a slow down of
all bitbake processes.

Thanks to Dongxiao Xu <dongxiao.xu@intel.com> for the great work in debugging
this. A large value has been left in for the read() command just to ensure some
fairness amongst process handling if a task tries to log truly huge amounts of
data to the server, or goes crazy and ensures the main loop doesn't stall.

(From Poky rev: 06c6db7929c75f576a395fb442abe447b833fc3b)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoAdd a few items to .gitignore
Chris Larson [Fri, 21 Jan 2011 01:51:52 +0000 (18:51 -0700)]
Add a few items to .gitignore

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agocache: fix docstring usage
Chris Larson [Thu, 20 Jan 2011 21:32:16 +0000 (14:32 -0700)]
cache: fix docstring usage

Use comments instead of docstrings where appropriate in CacheData.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agocooker: also respond to SystemExit for execute_runqueue
Chris Larson [Mon, 17 Jan 2011 15:50:40 +0000 (08:50 -0700)]
cooker: also respond to SystemExit for execute_runqueue

Without this, bitbake appears to hang when encountering an invalid task.  In
reality, the SystemExit flows up uncaught, the server may go away, and the UI
doesn't know about it, so requires us to ^C to exit.

This is a temporary measure.  Poky's bitbake handles invalid tasks
differently, and avoids this *particular* issue, but we do need to ensure that
the server responds to SystemExit from the idle function in a sane way.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/knotty.py: Ensure task note messages are only surpressed at default log levels
Richard Purdie [Wed, 12 Jan 2011 13:45:39 +0000 (13:45 +0000)]
bitbake/knotty.py: Ensure task note messages are only surpressed at default log levels

(From Poky rev: 3f08b7f1eadc8d181c76339d14c7abb4f49cad57)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agocodeparser: fix spacing in diagnostic messages
Bernhard Reutner-Fischer [Tue, 11 Jan 2011 18:21:55 +0000 (19:21 +0100)]
codeparser: fix spacing in diagnostic messages

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years ago*: use utils.remove() some more
Bernhard Reutner-Fischer [Mon, 10 Jan 2011 22:05:19 +0000 (23:05 +0100)]
*: use utils.remove() some more

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years agobuild: remove duplicate import of utils
Bernhard Reutner-Fischer [Mon, 10 Jan 2011 15:32:56 +0000 (16:32 +0100)]
build: remove duplicate import of utils

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
9 years agorunqueue.py: Call bb.cache.Cache.loadDataFull function directly when dumping signatur...
Richard Purdie [Mon, 10 Jan 2011 22:34:01 +0000 (22:34 +0000)]
runqueue.py: Call bb.cache.Cache.loadDataFull function directly when dumping signatures as per the rest of runqueue

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the BasicHash...
Richard Purdie [Mon, 10 Jan 2011 21:59:34 +0000 (21:59 +0000)]
bitbake/runqueue.py: Fix del_stamp calls and -f option to bitbake with the BasicHash siggen code

(From Poky rev: 48e567bb477ad468d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agomsg: no need to hardcode the logging levels
Chris Larson [Mon, 10 Jan 2011 19:30:05 +0000 (12:30 -0700)]
msg: no need to hardcode the logging levels

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: properly highlight python inside of ${@}
Chris Larson [Mon, 10 Jan 2011 19:00:41 +0000 (12:00 -0700)]
vim: properly highlight python inside of ${@}

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: handle highlighting exports without assignment
Chris Larson [Mon, 10 Jan 2011 19:00:09 +0000 (12:00 -0700)]
vim: handle highlighting exports without assignment

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovim: add an ftplugin for indentation settings
Chris Larson [Mon, 10 Jan 2011 18:57:59 +0000 (11:57 -0700)]
vim: add an ftplugin for indentation settings

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake: handle IOError from event_queue.get
Chris Larson [Mon, 10 Jan 2011 18:54:52 +0000 (11:54 -0700)]
bitbake: handle IOError from event_queue.get

These can occur in a particular ^C shutdown, so just break out of the event
queue empty loop if we encounter it.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoInject taskpid into log records via our log handler
Chris Larson [Mon, 10 Jan 2011 16:20:50 +0000 (09:20 -0700)]
Inject taskpid into log records via our log handler

It turns out that while log filters added with addFilter are only associated
with that logger, and not its children, handlers are inherited, and handlers
can be filters.   So, let's add filtering to our existing LogHandler class
which dispatches our log records as bitbake events.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoSwitch to fetch2 fetcher code based on the environment variable BBFETCH2
Richard Purdie [Mon, 10 Jan 2011 18:12:53 +0000 (18:12 +0000)]
Switch to fetch2 fetcher code based on the environment variable BBFETCH2

(From Poky rev: af06e4a8efbcaff657c397c34f2cb893d2d7f7f7)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.fetch2: replace bb.fetch with bb.fetch2 in the bb.fetch
Yu Ke [Mon, 10 Jan 2011 18:05:07 +0000 (18:05 +0000)]
bb.fetch2: replace bb.fetch with bb.fetch2 in the bb.fetch

bb.fetch2 is copied from bb.fetch, and has many bb.fetch referrence.
Fix these referrence with bb.fetch2 referrence

(From Poky rev: c64b445f38179a117d39319f762f9dc2435da0ea)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch2/git: Remove duplicated code only added for backwards compatibility
Richard Purdie [Mon, 10 Jan 2011 14:26:56 +0000 (14:26 +0000)]
fetch2/git: Remove duplicated code only added for backwards compatibility

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake: copy bb.fetch to bb.fetch2 as initial code base for fetcher overhaul
Yu Ke [Mon, 10 Jan 2011 14:25:04 +0000 (14:25 +0000)]
bitbake: copy bb.fetch to bb.fetch2 as initial code base for fetcher overhaul

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoutils.py: Update log message to use the logger calls
Richard Purdie [Mon, 10 Jan 2011 13:19:29 +0000 (13:19 +0000)]
utils.py: Update log message to use the logger calls

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoevent.py: Improve debug about invalid events
Richard Purdie [Mon, 10 Jan 2011 13:08:09 +0000 (13:08 +0000)]
event.py: Improve debug about invalid events

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agopersist_data: Don't loop forever waiting on database locks
Richard Purdie [Mon, 10 Jan 2011 13:00:39 +0000 (13:00 +0000)]
persist_data: Don't loop forever waiting on database locks

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignored
Richard Purdie [Thu, 6 Jan 2011 19:48:47 +0000 (19:48 +0000)]
bitbake/runqueue.py: Fix a bug where do_setscene dependencies would be ignored

(From Poky rev: df8569b4d89ce83e3cafd87f2f37b795d1bfbd6d)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agorunqueue.py: Fix undefined schedulers variable reference
Richard Purdie [Mon, 10 Jan 2011 12:53:11 +0000 (12:53 +0000)]
runqueue.py: Fix undefined schedulers variable reference

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agorunqueue.py: Add debug to help resolve partial log message failure
Richard Purdie [Mon, 10 Jan 2011 12:51:56 +0000 (12:51 +0000)]
runqueue.py: Add debug to help resolve partial log message failure

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agorunqueue.py: Add missing result return value in setscene failure path
Richard Purdie [Mon, 10 Jan 2011 12:51:00 +0000 (12:51 +0000)]
runqueue.py: Add missing result return value in setscene failure path

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agorunqueue.py: Fix up some lost range->xrange conversions misplaced as part of the...
Richard Purdie [Mon, 10 Jan 2011 12:50:10 +0000 (12:50 +0000)]
runqueue.py: Fix up some lost range->xrange conversions misplaced as part of the poky sync up

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agofetch/__init__.py: Add missing debug level argument
Richard Purdie [Mon, 10 Jan 2011 12:17:14 +0000 (12:17 +0000)]
fetch/__init__.py: Add missing debug level argument

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/siggen.py: Add a signature generator which adds the hash to the stamp files
Richard Purdie [Fri, 7 Jan 2011 11:04:38 +0000 (11:04 +0000)]
bitbake/siggen.py: Add a signature generator which adds the hash to the stamp files

(From Poky rev: 97789a160ad21ccc614f976fa5e2b306d816aabf)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake build/siggen/runqueue: Fix stampfile parameters
Richard Purdie [Thu, 6 Jan 2011 19:47:00 +0000 (19:47 +0000)]
bitbake build/siggen/runqueue: Fix stampfile parameters

The current parameters are not useful to the stampfile generator function
as they can't uniquely define a task. This updated things so the
parameters can identify unique tasks.

(From Poky rev: 66c6200ff34e8eeca5d1a689bbf9d6a83818248f)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobb.utils: check if lock file is writable, to fix Yocto bug 606
Yu Ke [Wed, 29 Dec 2010 07:28:48 +0000 (15:28 +0800)]
bb.utils: check if lock file is writable, to fix Yocto bug 606

Bug 606 report that if $DL_DIR is read-only, do_fetch will
simply hang without any error message.

The root cause is that: bb.fetch.go()->bb.utils.lockfile()
will try to lock file ${DL_DIR}/xxxxx.lock. Since ${DL_DIR}
is read-only, it will cause IOError exception. Although
lockfile() can catch the exception, currently code simply
ignore all the exception and continue the loop. it make
sense if the exception is caused by locking contention,
but in the read-only $DL_DIR case, it cause endless waiting
unfortunately.

So this patch add read-only check for lockfile to avoid the
silent hang.

(From Poky rev: 6ee0c26e21f48dcd47af88ed5c174e76e76a3e42)

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/knotty.py: Don't show log messages of NOTE and below for task processes
Richard Purdie [Fri, 7 Jan 2011 23:54:42 +0000 (23:54 +0000)]
bitbake/knotty.py: Don't show log messages of NOTE and below for task processes

This cleans up the knotty console messages to be a lot quieter and cleaning,
in keeping with the expectations of most users.

(From Poky rev: b22e345e05efcc3f66278af8f09fb083afe32b68)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/__init__.py: Add taskpid to all LogRecords (subclassed to be BBLogRecords)
Richard Purdie [Fri, 7 Jan 2011 23:52:19 +0000 (23:52 +0000)]
bitbake/__init__.py: Add taskpid to all LogRecords (subclassed to be BBLogRecords)

This allows us to identify which task messages are from.

(From Poky rev: f8e7215f6c4ece8c0a74ceee8da707cf791038e8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/msg.py: Add named definitions for all logging levels
Richard Purdie [Fri, 7 Jan 2011 23:50:17 +0000 (23:50 +0000)]
bitbake/msg.py: Add named definitions for all logging levels

(From Poky rev: a13352cfaf4b5d19196151e362851b1516e6af15)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoMerge branch 'poky-sync'
Chris Larson [Fri, 7 Jan 2011 15:51:43 +0000 (08:51 -0700)]
Merge branch 'poky-sync'

* poky-sync:
  build: kill unneeded environment setup bits
  parse: save python functions into the metadata
  Include exported variables in task process env
  runqueue: start implementing quieterrors
  Overhaul environment handling
  runqueue: resurrect use of file objects for pipein/pipeout
  runqueue: clean up message log levels
  runqueue: use correct task ID when checking validity of setscene tasks
  Add support for 'noexec' tasks
  runqueue: implement cache checking from metadata
  runqueue: pass task hash information to tasks
  runqueue: avoid unnecessary delays
  Implement task signatures
  data: Add emit_func and generate_dependencies
  codeparser: Implement persistent cache
  data_smart: track variable references
  codeparser: add module for parsing shell/python
  runqueue: implement scenequeue
  fetch: add SRC_URI checksum

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agocache: don't expand variables for skipped recipes
Chris Larson [Fri, 7 Jan 2011 15:38:41 +0000 (08:38 -0700)]
cache: don't expand variables for skipped recipes

Errors can result from these expansions, but for skipped recipes, we
shouldn't care about those failures.  This fixes the same issue which
Richard Purdie fixed in poky, commit 847b717.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/build.py: Fix del_stamp work correctly after recent stamp function changes
Richard Purdie [Fri, 7 Jan 2011 14:03:27 +0000 (14:03 +0000)]
bitbake/build.py: Fix del_stamp work correctly after recent stamp function changes

(From Poky rev: 5da9747)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/data.py: corrected the output for shell syntax.
Lianhao Lu [Fri, 7 Jan 2011 06:17:10 +0000 (14:17 +0800)]
bitbake/data.py: corrected the output for shell syntax.

[BUGID# 645], modify the emit_var()
1. Added "#" to the beginning of each line if the comment contains
multiple lines.

2. Added "\" to the end of each line if the shell variable value
contains multiple lines.

(From Poky rev: 6f454c10bcdd5)

Signed-off-by: Lianhao Lu <lianhao.lu@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake build.py: Stamp handling improvements
Richard Purdie [Thu, 6 Jan 2011 19:43:41 +0000 (19:43 +0000)]
bitbake build.py: Stamp handling improvements

* Move stamp file deletion out of the internal stamp helper function
* Add a new function to return the path to a stamp for a given task

(From Poky rev: ec6e4793dc79c61b780b74571db4cd66c1c04251)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake build.py: Use localdata for stamp handling, not d
Richard Purdie [Thu, 6 Jan 2011 19:40:41 +0000 (19:40 +0000)]
bitbake build.py: Use localdata for stamp handling, not d

(From Poky rev: 4a10a5b438f49577fb29f4e277e2300cdce10ab8)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agobitbake/build.py: Ditch the extract_stamp function as its just going to complicate...
Richard Purdie [Thu, 6 Jan 2011 15:41:39 +0000 (15:41 +0000)]
bitbake/build.py: Ditch the extract_stamp function as its just going to complicate improving the stamp handling functions

(From Poky rev: 51a2e8712a9b94de53253e198c64c0fa428e4349)

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
9 years agoparse: pass filename, lineno into the ast
Chris Larson [Tue, 4 Jan 2011 20:34:08 +0000 (13:34 -0700)]
parse: pass filename, lineno into the ast

We will be needing this information to improve the tracebacks of python code
from the metadata, as well as to give the user information about where
variables were defined, so they know how it ended up the way it is.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoparse.ast: drop __word__ regular expression
Chris Larson [Tue, 4 Jan 2011 20:07:27 +0000 (13:07 -0700)]
parse.ast: drop __word__ regular expression

We can use the string split method for this instead.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoparse.ast: avoid code duplication for inherit
Chris Larson [Tue, 4 Jan 2011 20:06:49 +0000 (13:06 -0700)]
parse.ast: avoid code duplication for inherit

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoprocess: simplify the log option for run
Chris Larson [Tue, 4 Jan 2011 20:03:24 +0000 (13:03 -0700)]
process: simplify the log option for run

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoevent: use BBLogFormatter in print_ui_queue
Chris Larson [Wed, 5 Jan 2011 18:20:05 +0000 (11:20 -0700)]
event: use BBLogFormatter in print_ui_queue

This ensures that when a failure occurs very early on in bitbake startup, the
message formatting ematches that used by the UIs.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobuild: use logger.isEnabledFor, not getEffectiveLevel
Chris Larson [Wed, 5 Jan 2011 18:19:21 +0000 (11:19 -0700)]
build: use logger.isEnabledFor, not getEffectiveLevel

I think this is less confusing, and avoids needing to know about the *range*
of logging levels, instead simply asking what we really want to know.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoAllow %20 in a file name in the SRC_URI origin/1.10
C Michael Sundius [Mon, 2 Aug 2010 21:56:11 +0000 (14:56 -0700)]
Allow %20 in a file name in the SRC_URI

Signed-off-by: C Michael Sundius <msundius@sundius.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobuild: kill unneeded environment setup bits
Chris Larson [Thu, 30 Dec 2010 01:47:41 +0000 (18:47 -0700)]
build: kill unneeded environment setup bits

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agovercmp: don't choke on empty first version components
Chris Larson [Fri, 26 Nov 2010 21:11:06 +0000 (14:11 -0700)]
vercmp: don't choke on empty first version components

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoDon't show tracebacks for SystemExit or KeyboardInterrupt
Chris Larson [Fri, 10 Sep 2010 18:33:48 +0000 (11:33 -0700)]
Don't show tracebacks for SystemExit or KeyboardInterrupt

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoFix exit code display for task failure
Chris Larson [Fri, 10 Sep 2010 18:14:54 +0000 (11:14 -0700)]
Fix exit code display for task failure

Per the python documentation, os.waitpid returns the exitcode shifted up by 8
bits, and we weren't compensating, resulting in a display of 'failed with 256'
when a worker process exits with a code of 1.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobitbake/server: Raise a shutdown event if we're seeing exceptions
Richard Purdie [Fri, 20 Aug 2010 08:38:24 +0000 (09:38 +0100)]
bitbake/server: Raise a shutdown event if we're seeing exceptions

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoConfHandler: commentary typo fixes
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:24 +0000 (20:57 +0100)]
ConfHandler: commentary typo fixes

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoevent: fix unicode handler registration
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:23 +0000 (20:57 +0100)]
event: fix unicode handler registration

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agocache: defer marking fn as clean
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:22 +0000 (20:57 +0100)]
cache: defer marking fn as clean

Only mark fn as clean if it is clean.
This saves us from removing (prematurely added) fn from our clean set
and saves me a few percent of runtime (and misleading debugging output
from remove()).

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoBBHandler: use basename directly in handler()
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:21 +0000 (20:57 +0100)]
BBHandler: use basename directly in handler()

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agodata: fewer newlines for (un)export
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:20 +0000 (20:57 +0100)]
data: fewer newlines for (un)export

Previously we emitted two newlines for export and unexport.
One newline for export and unexport is enough (and makes the scripts
look better and a tad smaller).

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoparse: Use constants from stat instead of magic numbers
Bernhard Reutner-Fischer [Mon, 3 Jan 2011 19:57:19 +0000 (20:57 +0100)]
parse: Use constants from stat instead of magic numbers

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoparse: save python functions into the metadata
Richard Purdie [Thu, 26 Aug 2010 17:06:30 +0000 (18:06 +0100)]
parse: save python functions into the metadata

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agoInclude exported variables in task process env
Chris Larson [Wed, 22 Dec 2010 01:23:22 +0000 (20:23 -0500)]
Include exported variables in task process env

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agorunqueue: start implementing quieterrors
Chris Larson [Wed, 22 Dec 2010 18:18:00 +0000 (13:18 -0500)]
runqueue: start implementing quieterrors

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoOverhaul environment handling
Richard Purdie [Sun, 28 Nov 2010 17:39:09 +0000 (17:39 +0000)]
Overhaul environment handling

Currently, anything whitelisted in the environment makes it into the worker
processes. This is undesireable and the worker environment should be as
clean as possible. This patch adapts bitbake sosme variables are loaded into
bitbake's datastore but not exported by default. Any variable can be exported
by setting its export flag.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agorunqueue: resurrect use of file objects for pipein/pipeout
Chris Larson [Mon, 20 Dec 2010 20:23:13 +0000 (15:23 -0500)]
runqueue: resurrect use of file objects for pipein/pipeout

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agorunqueue: clean up message log levels
Richard Purdie [Fri, 17 Dec 2010 10:45:51 +0000 (10:45 +0000)]
runqueue: clean up message log levels

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agorunqueue: use correct task ID when checking validity of setscene tasks
Kevin Tian [Tue, 14 Dec 2010 11:21:24 +0000 (19:21 +0800)]
runqueue: use correct task ID when checking validity of setscene tasks

sstate hash validation is done at initialization of RunQueueExecuteScenequeue.
However the index of 'valid' list returned from the validation doesn't
correspond to setscene task ID. It's just an intermediate namespace between
runqueue and sstate hash func. Use it as setscene task ID fully mess the flow.
Previously this doesn't cause trouble because all setscene tasks are passed. Commit
58396a5d24c62710fd0a9f3780d84ac8a95d8e7c add 'noexec' concept to setscene
tasks which grabs some tasks out of the list and thus trigger this problem

Without this fix there're ~50 recipes (gzip-native, glib, ...) rebuilt weirdly
with a minimal build, even though existing sstate packages could accelerate them.

there's another typo using wrong task ID in a debug message which further hide
this issue

Signed-off-by: Kevin Tian <kevin.tian@intel.com>
9 years agoAdd support for 'noexec' tasks
Richard Purdie [Sat, 6 Nov 2010 14:33:05 +0000 (14:33 +0000)]
Add support for 'noexec' tasks

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agorunqueue: implement cache checking from metadata
Richard Purdie [Tue, 5 Oct 2010 21:21:34 +0000 (22:21 +0100)]
runqueue: implement cache checking from metadata

This lets us speed up setscene processing, by letting the function from the
metadata inform us about the necessity of running the setscene functions.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agorunqueue: pass task hash information to tasks
Richard Purdie [Mon, 27 Sep 2010 14:57:13 +0000 (15:57 +0100)]
runqueue: pass task hash information to tasks

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agorunqueue: avoid unnecessary delays
Richard Purdie [Mon, 13 Sep 2010 15:57:13 +0000 (16:57 +0100)]
runqueue: avoid unnecessary delays

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoImplement task signatures
Richard Purdie [Tue, 31 Aug 2010 13:49:43 +0000 (14:49 +0100)]
Implement task signatures

Includes functionality to find out what changes between two different
signature data dumps.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agodata: Add emit_func and generate_dependencies
Richard Purdie [Tue, 3 Aug 2010 13:30:20 +0000 (14:30 +0100)]
data: Add emit_func and generate_dependencies

These functions allow generation of dependency data between functions and
variables allowing moves to be made towards generating checksums and allowing
use of the dependency information in other parts of bitbake.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agocodeparser: Implement persistent cache
Richard Purdie [Tue, 31 Aug 2010 13:19:22 +0000 (14:19 +0100)]
codeparser: Implement persistent cache

For a given input to this code, the output doesn't change to implement a persistent
cache of the data to speed up parsing.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agodata_smart: track variable references
Richard Purdie [Wed, 25 Aug 2010 13:26:37 +0000 (14:26 +0100)]
data_smart: track variable references

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agocodeparser: add module for parsing shell/python
Richard Purdie [Mon, 2 Aug 2010 09:20:20 +0000 (10:20 +0100)]
codeparser: add module for parsing shell/python

This commit is derived from Chris Larson's checksum work, turned into a
standalone piece of code for parsing python and shell functions.

The deindent code has been replaced with code to work around indentation
for speed. The original NodeVisitor in the ast was replaced with a faster
class walk call.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agorunqueue: implement scenequeue
Richard Purdie [Thu, 19 Aug 2010 10:36:29 +0000 (11:36 +0100)]
runqueue: implement scenequeue

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agofetch: add SRC_URI checksum
Yu Ke [Fri, 17 Dec 2010 05:52:52 +0000 (13:52 +0800)]
fetch: add SRC_URI checksum

This patch adds per-recipe SRC_URI checksum verification.

- SRC_URI format
The format of SRC_URI checksum follow OE definition:

1. SRC_URI has single src
SRC_URI = "http://some.domain/file.tar.gz"
SRC_URI[md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"

2. SRC_URI has multiple src, every src need specify name
SRC_URI = "http://some.domain/file1.tar.gz;name=name1 \
           http://some.domain/file2.tar.gz;name=name2 "
SRC_URI[name1.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name1.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"
SRC_URI[name2.md5sum] = "xxxxxxxxxxxxxxx"
SRC_URI[name2.sha256sum] = "xxxxxxxxxxxxxxxxxxxxxx"

- SRC_URI checking invocation:
the checksum checking is invoked in do_fetch phase,
so it can be invoked manually by

if recipes has no SRC_URI checksum item, bitbake will show warning:
"
WARNING: Missing SRC_URI checksum for xxxx.tar.gz, consider to add
SRC_URI[md5sum] = "5c69f16d452b0bb3d44bc3c10556c072"
SRC_URI[sha256sum] = "f4e0ada8d4d516bbb8600a3ee7d9046c9c79e38cd781df9ffc46d8f16acd1768"
"
thus recipe author can add it to recpie file after SRC_URI

- control variable BB_STRICT_CHECKSUM
when SRC_URI checksum is missing, this variable decide pass or not
if BB_STRICT_CHECKSUM = "1", missing checksum is fatal

Signed-off-by: Yu Ke <ke.yu@intel.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoprocess: fix handling of the input argument
Chris Larson [Thu, 30 Dec 2010 06:44:21 +0000 (23:44 -0700)]
process: fix handling of the input argument

When using a logfile, we weren't sending input to the child process.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agofetch: handle no sortable_buildindex for git
Chris Larson [Thu, 30 Dec 2010 01:39:50 +0000 (18:39 -0700)]
fetch: handle no sortable_buildindex for git

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agoutils: kill unnecessary duplicated chmod
Chris Larson [Thu, 30 Dec 2010 00:52:54 +0000 (17:52 -0700)]
utils: kill unnecessary duplicated chmod

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agofetch: fix git rev key compat for sortable
Chris Larson [Wed, 29 Dec 2010 23:46:21 +0000 (16:46 -0700)]
fetch: fix git rev key compat for sortable

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agobb.debug: handle caller not passing a debug level
Chris Larson [Tue, 28 Dec 2010 17:02:49 +0000 (10:02 -0700)]
bb.debug: handle caller not passing a debug level

This is a compatibility measure, as previous versions did not error out when
this occurred, and will most likely be dropped in the next bitbake minor
version in favor of a fatal error.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agofetch: resurrect branch inclusion in git revision key
Chris Larson [Wed, 29 Dec 2010 23:23:14 +0000 (16:23 -0700)]
fetch: resurrect branch inclusion in git revision key

This is similar to Richard Purdie's commit, but falls back to checking the
persist_data database for the old key if the new does not exist, so as to
retain compatibility for at least one bitbake minor version.

Signed-off-by: Chris Larson <chris_larson@mentor.com>
9 years agofetch: Ensure SRCREV is still set correctly if the pn-X override has been expanded
Richard Purdie [Sat, 9 Oct 2010 16:24:16 +0000 (17:24 +0100)]
fetch: Ensure SRCREV is still set correctly if the pn-X override has been expanded

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>
9 years agofetch: Fix named SRCREVS variable name ordering
Richard Purdie [Thu, 9 Sep 2010 14:09:50 +0000 (15:09 +0100)]
fetch: Fix named SRCREVS variable name ordering

Retains backward compatibility.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>