pysh: add missing os.path import

(Bitbake rev: 6fb448a6cfb00aeb5dfc6c047915e0941e754348)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

runqueue: fix typo in documentation of calculate_task_weights

(Bitbake rev: 5cde120003af97a5cf5c6eff2a02bb1480f9414b)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

runqueue: use fewer newlines in error path

(Bitbake rev: 7498466f0e42beea6f5f411209a892b636c7783c)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

logger usage cleanup

(Bitbake rev: 976e4f84a8147ad762442df7ff4820611a21d227)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

utils: fix typo in error message

(Bitbake rev: 99cdb61b30d0c75c8f831c78346cc9f8ca7945dd)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Move LAYERDIR expansion hack into DataSmart

(Bitbake rev: 40778a6e9e82c7ea4673a74fc19574430fa63e8d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

parse: use bb.utils.which

(Bitbake rev: 2bc893dc407bf608cac7e12673161b75b712da83)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix logging level names for post-server-ui-split

(Bitbake rev: dc5a5c39291ec223cd761dce59d29eee7316cb70)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Use os.devnull, not /dev/null

(Bitbake rev: ce5ab0fc524a1c2c48c4c39d6fc8aae23019207b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

utils: don't overwrite builtin dir() function

(Bitbake rev: 6dc863f714beda6ca7ff8cd3e830a9bc8a39123d)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bzr: use utils.remove instead of os.system

(Bitbake rev: 0ef8a9b3780525a28567ebe42956e5cae79ada8c)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

utils: drop unnecessary debug msgs from mkdirhier

(Bitbake rev: 4b36be445a05a544cd3a456acd1dcc8f962359e2)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Show the user progress when loading the cache

(Bitbake rev: bdd7813d8eecf7b6b636322e748ca6bf69118513)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

knotty: exit with 1 if we see a critical log message

(Bitbake rev: d527ca441539618c990291fb8340f552ac760bce)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: don't swallow recipe parse errors

(Bitbake rev: 1f5d49ce64e5e1c7705edbfa3c8e19649c21edd2)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

manual: document scmdata=keep for CVS, SVN and Git

(Bitbake rev: f42e6d33b986183eee2c74aa7b1bbc9b339276a9)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetchers: Add parameter scmdata=keep to include .git/ and others in generated tarballs.

* Allows generating version information from SCMs during build.
* Note that tar doesn't need to use --exclude '.git', because
  git checkout-index doesn't clone the repository.

(Bitbake rev: 05cbc1d1a01c667c77688f36fbc5b61c5f452a3a)

Signed-off-by: Andreas Oberritter <obi@opendreambox.org>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: fix FuncFailed for the no logfile case

(Bitbake rev: 3a4b5eb20e41bd3cecd9d7e8dff7d076d8a7bb9a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

process: handle OSErrors other than file not found

(Bitbake rev: 7d80a5355cb540aae8d3082c1efebb72da4f93c6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: ensure LogTee has a valid name property

(Bitbake rev: 0ebb46e25261cfc85aaef2790cba7c1ec057c306)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix PWD issue with new exec_func_shell

The previous attempt was incorrect.  The issue isn't that subprocess fails to
set PWD, it's that PWD is in the metadata, inherited from the environment, and
is re-exported, overwriting the actual accurate one in the shell environment
with the old one from the metadata.  So, ensure that PWD in the metadata is
not exported.

We can ditch this when the environment handling is reworked (e.g. poky's
commit to do so).

(Bitbake rev: 2c8683234acf514706b2b69f5b29405485e664dd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Ensure that the invalid UI error goes to stderr

(Bitbake rev: 76aeea2c23a518046d0b26f1a6fc43287a180a9d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: set PWD in the subprocess preexec hook

(Bitbake rev: efa5485c8f9d488ba058c40734cc55296d3de6eb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bitbake: Transfer noexec runqueue messages to the UI

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Move the runq task start/failed messages to the UI

(Bitbake rev: ab831e867f09b47001cb8da2f8f060e04febf237)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Silence python 2.7 nested context manager warning

(Bitbake rev: 1c38f19fb8da2e6c55394162d0142e7146387e54)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Rename command events, adjust compareRevisions

- Moved the logic for comparing revisions from cooker into command
- Removed 'Cooker' from the event names
- Renamed the 'ExitCode' event into CommandExit, and changed CommandFailed to
  be a subclass of CommandExit

(Bitbake rev: c51ed5d7a9971fad6019dac6c35a71b8a54ab16a)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

event: use cPickle for events

(Bitbake rev: b16c0c1dc3b7e03b02555e9803767a0d695e67f1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

utils: fix 'lock' variable reference

(Bitbake rev: f57f8f3cc980e3ae1693c2e065227e951ed0b1c0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bb.event: fix MsgBase ref in fire_class_handlers

(Bitbake rev: db7f960e5f103a424a4319f1867fb540e643c1ec)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Kill the uncaught exception handler

We now wrap the main() in a try/except, ensuring that both the main portion of
bin/bitbake and the UI raising an exception will be shown to the user.  For
the server and workers, we can ensure in the server itself that exceptions are
handled correctly.

(Bitbake rev: 240d4a7ae80a6636c302ae84266ddfed7a7fcedd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: fix -D with shell functions

(Bitbake rev: 1c8be64732fdf4f3a608c090b3dc92065d6058d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: send logging messages to the log file for python functions

(Bitbake rev: ee1cce6ab21ddda60a7a070d03e98ff8485a5e71)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

utils: fix calls to close() in the lock/unlock functions

(Bitbake rev: 4262c26d36d1c1b6801ac422716a227c1f6b9589)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: use a contextmanager for locks

Also don't bother passing logfile to exec_func_python, at least until we start
adding the logfile as a file handler to the bitbake logger.

(Bitbake rev: f99ee4680c9f67b7ed13fc06044ba2382f9a782c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

build: use bb.process instead of os.system

(Bitbake rev: 53740977521bc81ffa37adfa7bbeb8f2a80ea165)

build: write logfiles per task, not per function
Based on d14f9bf6 from poky, reworked for master and other cleanup.

(Bitbake rev: beadff2eca1eb95f0411115dd72ddb4c3c44c604)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

process: add subprocess-based bits

(Bitbake rev: c63e55564a8840083dbd8634b10fe6f76d1f1354)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Rename the ui 'init' method to 'main'

As these may run the UI in a blocking fashion and then return the exit code,
'init' was an inappropriate name, and 'main' is more appropriate.

(Bitbake rev: 4d081a0ed759bd526ab01849d650bd9e8d80ddd1)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: use re match, not search in re_match_strings

We want to match the requested pattern at the beginning of the string,
otherwise things behave in an unintuitive manner wrt ASSUME_PROVIDED (e.g.
ASSUME_PROVIDED += "gtk+" will also assume foo-gtk+ is provided), and the user
can always use '.*gtk+' to get the old behavior.

(Bitbake rev: 5670134ab2eb573d39df3c3231677cdb1a1dfc72)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

taskdata: use 'any' in re_match_strings

(Bitbake rev: e48e9a2150ee76aaf151f6d5bc9e86e6ae4de514)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: add shutdown/stop methods

(Bitbake rev: fc64eff03fe1f41e59128cb3df0ef2161e24a8cb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: merge cookerState and cookerAction

(Bitbake rev: c7c8945ef7ca9465312e630b7fa5f0a87ac8b6c7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: use a pool, abort on first parse error

(Bitbake rev: 9caf65e79f95fe0045e727391e974c4c1e7411ff)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

knotty: use enumerate for task waiting

(Bitbake rev: 89ce8df075ac8c9a5478c86405e6e6b60346a51c)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

knotty: shift non-interactive progress into a class

(Bitbake rev: c3d005cbbae3d56da9926666cfb1501c2bf96ea7)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix the <100 recipe progress fix

(Bitbake rev: 424428a764651183218f9cc93bc05496867aa5de)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Error more pleasantly when trying to use python2.4

(Bitbake rev: 57402ee727c8c1d8f5017534fb0f5e0b20aaaa5b)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Don't show a traceback for a python version error

(Bitbake rev: 44c325306e55336b7b416fd88b39874012a3f603)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: don't choke on <100 recipes to parse

(Bitbake rev: 2be5169a01e57bcfcb1f75cbd6bac3bf0f230edd)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: no cached in progressbar and add ETA

Rather than updating the progress bar based on the recipe being processed
(whether cached or parsed), consider only parsed recipes.  This reduces the
instability in progress rate introduced by the cached entries, and allows the
ETA to be resurrected and be a bit more useful.

(Bitbake rev: 618480f7739f6ae846f67a57bee5a78efb37839d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: change to more incremental format

(Bitbake rev: 4fe4ffbef3885887c97eebe021edc3f23feab9ea)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: pass back child exceptions to the server

(Bitbake rev: 0f68f8bcd0e0aa944f76f88a4a85c9bcc1e42bee)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

vercmp: don't choke on empty first version components

(Bitbake rev: e75aa94e9477933c5a40021b2a8e844db54f29da)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Use __file__, not sys.argv[0]

(Bitbake rev: 97e92abe49663eee189c89c1dc91fe69891faf73)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: ensure 'pn' is included in the pkgvars

(Bitbake rev: cefc28a2aa1e5703119dacfd885c8c159a1e47a3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

knotty: drop the ETA from the progressbar for now

Currently, the progress bar is an indication of the processing of our recipes,
which includes loading the cache file, then for each recipe, either adding the
existing cached information to the CacheData or parsing the recipe from disk.
These tasks clearly take different amounts of time, so the ETA is unreliable
today.  We'll resurrect this functionality after we revamp the progress
handling, fully incorporating the load of the cache file.

(Bitbake rev: 80867372dcbef91ebaf7d77a77ca871741dd3f74)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: ensure that the cache sync completes

Without explicitly joining the thread, it's possible for the process to end
(e.g. after a bitbake -p) and kill off the thread without waiting for it to
exit cleanly.  So, register the thread join with atexit.

(Bitbake rev: 97ce57e6f860d3e6f34cc7a603ed1eeac4f423d3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: get number of threads in constructor

(Bitbake rev: e7fd259d1528f48b0812fb7b0a54dd6d5d22f4a9)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: don't add info for skipped recipes

(Bitbake rev: ac596d55de4ad86e1cdc84dabfea1350dd54efd0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: show progress bar before initializing the cache

This ensures that the time spent loading the cache from disk occurs with the
progress bar up.  Though the progress bar stays at 0% during this period, I
think this is an improvement over the multi-second stall which occurred
previously before the progress bar came up.  Ideally, we'd integrate cache
loading from disk into the progress display, but this is a first step.

(Bitbake rev: f6d0a5c219f9deb84f702450d30d868ba6271f77)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: save progress chunk value (total/100)

(Bitbake rev: 09333737cbeeb9875d938521ddcd519fc808bcc3)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: stop loading the cache for -b

Previously, the cache was actually being loaded from disk twice whenever using
-b or -e -b.  This also moves the bb_cache instance into the CookerParser, as
it's not needed by the cooker itself at all.

(Bitbake rev: dd0ec2f7b18e2a9ab06c499b775670516bd06ac8)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cooker: don't fire unnecessary parse progress events

(Bitbake rev: 7cf7d5f14405681496fced3640a50a20ef1acac1)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: don't add info to cache if cache is disabled

(Bitbake rev: f12bb303f3d86a68d0b3dda1112dd654b9251704)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: sync the cache file to disk in the background

This version uses a thread rather than a process, to avoid problems with
waitpid handling.  This gives slightly less overall build time reduction than
the separate process for it did (this reduces a -c compile coreutils-native by
about 3 seconds, while the process reduced it by 7 seconds), however this time
is quite insignificant relative to a typical build.

The biggest issue with non-backgrounded syncing is the perceived delay before
work begins, and this resolves that without breaking anything, or so it seems.

(Bitbake rev: 5ab6c5c7b007b8c77c751582141afc07c183d672)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Simplify cache syncing

Rather than adding nocache items to the cache, then copying the cache and
removing them to sync it, don't add them in the first place.  Also use 'with'
for the cachefile.

(Bitbake rev: 343b6f6255ad020c39e30742175a241f0859a5a6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Experimental usage of the 'progressbar' module

(Bitbake rev: 64feb03bc2accecb49033df65e0a939ef5ab5986)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Implement parallel parsing support

This utilizes python's multiprocessing module.  The default number of threads
to be used is the same as the number of available processor cores, however,
you can manually set this with the BB_NUMBER_PARSE_THREADS variable.

(Bitbake rev: c7b3ec819549e51e438d293969e205883fee725f)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: don't write out the cache unnecessarily

If the only recipes's we reparsed this run were those flagged as not to be
cached, there's no point in re-saving the cache, as those items won't be
included anyway.

(Bitbake rev: 1e0c4dbcbec886a30b89f8b4bb365c3c927ef609)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: create and use a RecipeInfo class

This class holds the particular pieces of information about a recipe which are
needed for runqueue to do its job.

By using it, I think we improve code clarity, reduce method sizes, reduce
overuse of primitive types, and prepare for parallel parsing.  In addition,
this ditches the leaky abstraction whereby bb.cache attempted to hide the
difference between cached data and a full recipe parse.  This was a remnant
from the way things used to be done, and the code using it had to know the
difference anyway.  If we choose to reimplement caching of the full recipes,
we can do it in bb.parse, in a completely transparent way.

(Bitbake rev: 992cc252452221f5f23575e50eb67528b2838fdb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Prefer xrange over range for small performance gain.

range() allocates an actual list when called.  xrange() is just an iterator
and creates the next range item on demand.  This provides a slight
performance increase.

In python 3, range will do what xrange does currently, but the upgrade will
be handled by the 2to3 tool.

(Bitbake rev: 73b40f06444cb877a5960b2aa66abf7dacbd88f0)

Signed-off-by: Bob Foerster <robert@erafx.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Revert "svn fetcher: warn people to switch to SRCREV"

1) too spammy
2) can be implemented in the metadata instead

This reverts commit 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b.

Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: make loadDataFull a classmethod

(Bitbake rev: 16a9d689e61c35cfca94bbecd9772eab3c5072ba)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

svn fetcher: warn people to switch to SRCREV

as noted by rp in ac00ca89a4e43cd4f38ba86455079d31be78e644

(Bitbake rev: 8da9744fcdf856abebcfbe9e3bc1b8cf07bc317b)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetch: be more pythonic

no functional changes

(Bitbake rev: e88834fb7c6821cc29c12d296f2edd51f6eb3746)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetch: use os.path.join

(Bitbake rev: c360b01df18d90a513a3d61d395f905102e7568e)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetch: add common helper _strip_leading_slashes()

Several fetcher need a way to strip leading slashes off a local path.
This helper-function consolidates all such occurances.

(Bitbake rev: 823a02185ed109054c6c1ae366221aaed0353f24)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetch: rename suppports_srcrev to supports_srcrev

osc had it already spelled correctly?!

(Bitbake rev: b8bb4433de7a981c6826173e926ca34705c4ac70)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: Add debug msg for a nonexistant dep file

If a recipe depends on a file, and that file is out of date, we show a
message, but if that file was removed, we do not, until now.

(Bitbake rev: 67984ba0ac2db79874541bc031f2e3e9ff7a6c32)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: use new style classes

(Bitbake rev: e084430446be2544dd1a6b627088f888c37cc7f0)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: pyflakes/pep8/pylint cleanup

(Bitbake rev: 06420ff839ffc37de3e42474b8b0e47c4608a985)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: use set for clean, checked

(Bitbake rev: 3a776ef947e23b1a04bdd9e78a3a0c4616249b0e)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

cache: make use of defaultdict

(Bitbake rev: 7171019b11ad656d0edb979564941fcf92ab0a02)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Pre-explode rundeps/runrecs in CacheData

(Bitbake rev: 0c8e0f3191252ccedb83c431c48e5c7d905e2717)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

persist_data: handle locked db for SELECT

Parallel processes interacting with the persist_data db can quite easily
explode without this.

(Bitbake rev: b3d5432cff0ff28f4c8a5bcf10efa3e383b4fd4d)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

BBHandler: use os.path in inherit()

(Bitbake rev: 5b85de2c71973ba490b95a5d9ab634635f395142)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

BBHandler: simplify supports()

(Bitbake rev: 858d704d713d15bf97053eb1374758c74b0d8874)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bb.cache: only log if the respective action was taken

This avoids alot of misleading log-messages like "Removing FOO from cache"
if FOO was not in the cache and as such is not a removal candidate.

(Bitbake rev: de34a403e206867e09410ad4925c7b9cff04fee6)

Signed-off-by: Bernhard Reutner-Fischer <rep.dot.nop@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bb.pysh: add Case support to format_commands

(Bitbake rev: 6e24f573a0e95068eb9237c1d264ad1148b2f690)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bb.pysh: fix writing pyshtables all over the place

(Bitbake rev: ff4753e362714a3c4c759c2fad8a9e5b8fe5bef5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Move the pysh package into the bb package

The pysh we're using is modified, and we don't want to risk it conflicting
with one from elsewhere.

(Bitbake rev: 1cbf8a9403b4b60d59bfd90a51c3e4246ab834d6)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Allow mercurial fetcher to follow tip

There are occasions when developing when I want a package always to
grab the latest copy of a package.  Witht eh CVS fetcher you can do
this by setting the `date' tag to `now'.  This patch adds similar
functionality to the mercurial fetcher: if the revision to fetch is
`tip' then always grab from the server, and don't use the cached
tarball.

Oh, and I fixed a typo in the Class comment.

(Bitbake rev: 01b85608d8a37f8af66dfd80133e950120679079)

Signed-off-by: Peter Chubb <peter.chubb@nicta.com.au>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix GraphViz .dot output for rdepends and rrecs

Use bb.utils.explode_deps to break up the rdepends and rrecommends strings.
This fixes the same issue which was fixed by a number of patches floating
around, but uses explode_deps rather than regular expressions.

(Bitbake rev: 83cdb23f8b89453a3527a276bd0b4deb85d63deb)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Make 'cache is clean' message debug

(Bitbake rev: 92637a355d55cb66de91b4314bc0e7cf1ac64ade)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

bitbake: lib/bb/fetch/hg: fix fetching from a mercurial repository

* without this fix, we get :
updating working directory
74 files updated, 0 files merged, 0 files removed, 0 files unresolved
abort: There is no Mercurial repository here (.hg not found)!

(Bitbake rev: 75ea005ac8fc05b2b3afca803d77a6b5f558efee)

Signed-off-by: Eric Bénard <eric@eukrea.com>
Tested-by: Frans Meulenbroeks <fransmeulenbroeks@gmail.com>
Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Resurrect display of failed files

This was inadvertantly removed when trying to reduce the amount of duplicated
information the user sees when a failure occurs.

(Bitbake rev: 850d6158ea9daa58e896fd6b258d586df797dcf4)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Access metadata vars as locals in python snippets

Example:
FOO = "bar"
BAR = "${@FOO + '/baz'}"

${BAR} == "bar/baz"

(Bitbake rev: 606fa1fd97cbd47a6a7ebdc7a2e6aa93a8f65cf5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix __getitem__ for DataSmart

Ensure it raises KeyError for a missing key, this is required to use this as a
mapping in various places, e.g. as locals in an eval.

(Bitbake rev: 8d661ce0c303e8d69f17c1d095545d5ed086d1d5)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

fetchers: Use tar --exclude pattern to remove SCM files

This option will exclude the SCM metadata from tar files.

Tested with gcc where svn tar which used to be 156M for gcc 4.5
is now 77M

(Bitbake rev: f264cb6d43472525ad787b0887764ea696ec52ba)

Signed-off-by: Khem Raj <raj.khem@gmail.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>

Fix syntax issue and don't include 'd' in ui_queue

(Bitbake rev: bed8e09971dc577f5443ad3d89aa14634c54eb16)

Signed-off-by: Chris Larson <chris_larson@mentor.com>
Signed-off-by: Richard Purdie <rpurdie@linux.intel.com>