Elliot Smith [Tue, 12 Jul 2016 22:54:27 +0000 (15:54 -0700)]
toaster.bbclass: remove directory scan logic for detecting artifacts
toaster.bbclass does a scan of the image deploy and SDK directories
when a build finishes. However, this brings no benefit and could be
better managed and made easier to modify if moved to toasterui and
carried out when the BuildCompleted event occurs.
Remove the image scan code from toaster.bbclass, prior to moving it
to toasterui and buildinfohelper.
Also remove the license manifest update code, as this can also be
done from toasterui.
The postfuncs for do_populate_sdk are retained, but no longer
do the directory scan for SDK artifacts. Instead, they fire
an event with the value of the TOOLCHAIN_OUTPUTNAME variable,
as this is only accessible at the point when the do_populate_sdk
and do_populate_sdk_ext tasks are run. The value of this can then
be used by buildinfohelper to find the SDK artifacts produced by a
target.
[YOCTO #9002]
Signed-off-by: Elliot Smith <elliot.smith@intel.com> Signed-off-by: bavery <brian.avery@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
In Python3 the itertools module's imap function has been migrated to the
globalname space as map(). Calling itertools.imap() will fail because it
no longer exists.
Signed-off-by: George McCollister <george.mccollister@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
buildstats-summary: round the floating point percentage
This was rounded in python 2, but python 3 changed the default behavior of /.
We could switch to the same behavior as previous by switching to // rather
than /, but there's value in keeping at least one decimal point, to avoid the
misleading case where it says 0% but the reuse is non-zero.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:57:09 +0000 (11:57 +1200)]
lib/oe/patch: handle encoding differences in patch files
With Python 3, the encoding of a file is significant; several recipes in
OE-Core have patches which are not fully utf-8 decodable e.g. man,
lrzsz, and gstreamer1.0-libav, leading to errors when using devtool's
modify, upgrade or extract subcommands on these recipes. To work around
this, try reading the patch file as utf-8 first and if that fails try
latin-1 before giving up.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:57:08 +0000 (11:57 +1200)]
recipetool: create: support specifying a file as the local source
It is currently possible to specify a file (e.g. a tarball) on the local
disk as the source, but you have to know to put file:// in front of it.
There's really no need to force users to jump through that hoop if they
really want to do this so check if the specified source is a file and
prefix it with file:// if that's the case.
Also ensure the same works for "devtool add" at the same time.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:57:07 +0000 (11:57 +1200)]
recipetool: create: fix handling of github URLs
For a while now, Github hasn't been advertising a specific repository
URL since cloning the web URL with git works. Armed with this knowledge
and fully expecting people to just paste the github URL, we need to
handle this situation specially. If it looks like a github URL to the
root of a repository then treat it as a git repository instead of a
normal https URL to be fetched by the wget fetcher.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
If devtool returns exit code 4 then record the recipes as "skipped"
rather than "failed" - these are recipes we know cannot work (usually
because they don't provide any source).
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Sun, 10 Jul 2016 23:07:56 +0000 (11:07 +1200)]
devtool: return specific exit code for incompatible recipes
Certain recipes cannot be used with devtool extract / modify / upgrade -
usually because they don't provide any source. Return a specific exit
code (4) so that scripts such as scripts/contrib/devtool-stress.py know
the difference between this and a genuine failure.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:57:11 +0000 (11:57 +1200)]
devtool: reset: allow reset to work if the recipe file has been deleted
We were attempting to open the recipe file unconditionally here - we
need to account for the possibility that the recipe file has been
deleted or moved away by the user.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:57:10 +0000 (11:57 +1200)]
devtool: update-recipe: fix --initial-rev option
In OE-Core revision 7baf57ad896112cf2258b3e2c2a1f8b756fb39bc I changed
the default update-recipe behaviour to only update patches for commits
that were changed; unfortunately I failed to handle the --initial-rev
option which was broken after that point. Rework how the initial
revision is passed in so that it now operates correctly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
The Egyptian government changed its mind on short notice, and
Africa/Cairo will not introduce DST starting 2016-07-07 after all.
(Thanks to Mina Samuel.)
Asia/Novosibirsk switches from +06 to +07 on 2016-07-24 at 02:00.
(Thanks to Stepan Golosunov.)
Changes to past and future time stamps
Asia/Novokuznetsk and Asia/Novosibirsk now use numeric time zone
abbreviations instead of invented ones.
Changes affecting past time stamps
Europe/Minsk's 1992-03-29 spring-forward transition was at 02:00 not 00:00.
(Thanks to Stepan Golosunov.)
Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Robert Yang [Thu, 7 Jul 2016 01:37:04 +0000 (18:37 -0700)]
libpcre: 8.38 -> 8.39
* Remove CVE-2016-3191.patch which is already in the source.
* The LIC_FILES_CHKSUM is changed since it has updated the date from
2015 to 2016, the contents are the same.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Richard Purdie [Fri, 8 Jul 2016 17:00:49 +0000 (18:00 +0100)]
linux-yocto-dev: Handle performance regression
After commit 0437a59e3c298d40aaa96af09b80bff8fcbe292d, the linux-yocto-dev
recipe is being parsed every time we run "bitbake -p". This was spotted
on some performance benchmarks and showed up as a performance regression.
We can tweak the recipe to ensure this doesn't happen and that its only
used if selected.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 6 Jul 2016 23:48:34 +0000 (11:48 +1200)]
gcc: add runtime packages to RRECOMMENDS
In order to use certain features of gcc, you need the corresponding
runtime library. It seems to me that these ought to be installed by
default when installing the compiler since they are required if certain
command line options are used, so add them to RRECOMMENDS. I used
RRECOMMENDS since some of these packages may or may not exist depending
on architecture and build options; additionally it makes it possible to
use BAD_RECOMMENDATIONS if you really want to exclude them.
The impact of this isn't too bad in the context of an image providing
on-target compilation - about a 30MB increase in size for an image
containing gcc and g++.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Mon, 11 Jul 2016 21:03:38 +0000 (22:03 +0100)]
glibc: add more Imagination Meta relocation to elf.h
glibc master added the EM_METAG tag but didn't add the relocation defines.
However the kernel tooling only checks for EM_METAG when defining its own values
so scripts/recordmcount ends up using R_META_* symbols without their definition.
Whilst the kernel can and should be fixed, this breaks all users of recordmcount
so patch elf.h to add the values.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Zhenhua Luo [Tue, 28 Jun 2016 07:25:07 +0000 (15:25 +0800)]
tune-ppce500mc.inc: pass -mcpu=e500mc for ppce500mc kernel compile
Currently the -mcpu parameter is not passed to cross gcc when assembling
kernel .S file, the implicit -mcpu option that defaults to the latest
server cpu might casuse incorrect assembling.
A existent case is that wait instruction of ppce500mc is incorrectly assembled
to power9 version with default -mcpu setting, accordingly kernel boot calltrace
happend when wait instruction is executed on ppce500mc targets.
Signed-off-by: Zhenhua Luo <zhenhua.luo@nxp.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Tue, 5 Jul 2016 16:41:24 +0000 (17:41 +0100)]
classes/libc-package: remove pointless copying when running localedef
localedef handles attempts to read/write the archive in parallel correctly by
creating the file atomically, gracefully handling racing to create, and has
exclusive locks when writing. Therefore I can't see any purpose to copying the
archive to /tmp and back again when manipulating it.
Signed-off-by: Ross Burton <ross.burton@intel.com>
This fragment dates from when this class was used for more than just glibc
locale packaging, and as glibc-locale disables do_configure it can't have been
executed.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Tue, 23 Feb 2016 14:05:35 +0000 (14:05 +0000)]
db: use S/B more idiomatically
Instead of setting S to a directory inside the tarball and B to another
directory inside the tarball, use the default value of S and set
AUTOTOOLS_SCRIPT_PATH to the right path to find configure. Update the patches
so they still apply, and clean up the recipe slightly.
Because something is not quite right regarding quilt and patching, add a PR bump
to the recipes to ensure that a clean work directory is used: for some reason
rebuilds will rarely fail to patch correctly.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Add a --disable-debug-info parameter to ./configure, as it otherwise
attempts to execute a small test binary. If this (new) feature is truly needed,
we can set up a qemu wrapper later, or patch autoconf macros.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Richard Purdie [Wed, 6 Jul 2016 16:21:59 +0000 (17:21 +0100)]
terminal: Drop gnome-terminal --disable-factory workarounds
With the new pid monitoring code we have for recent versions of
gnome-terminal we can just drop the --disable-factory code now since
the other solution handles this case as well.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 6 Jul 2016 16:08:57 +0000 (17:08 +0100)]
terminal: Fix gnome-terminal to work with recent versions
Currently gnome-terminal just returns straight away, opening a terminal in a new
separate process we have no insight into. For patch resolution, this leads to
spawning many different terminal windows, for pydevshell, it just flashes a window
up and then closes.
We need to block until the command completes but gnome-terminal gives us no way
to do this. We therefore write the pid to a file using a "phonehome" wrapper
script, then monitor the pid until it exits.
[YOCTO #7254]
(also fixing do_devpyshell)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 5 Jul 2016 12:18:31 +0000 (13:18 +0100)]
pseudo: Upgrade to 1.8.1
* Drop patches where the changes exist upstream
* Fetch from git as no tarball is available for 1.8.1
* Move common code to pseudo.inc
* Update patchset in git recipe
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 7 Jul 2016 21:17:32 +0000 (09:17 +1200)]
oe-selftest: drop test_prepare_unexpected
This test refers to a function that no longer exists after the eSDK
install double execution of bitbake has been removed, and since
test_prepare_unexpected is the only test in this module, drop the
entire module. We can easily resurrect it if we have unit tests to add
in the future.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Igor Stoppa [Fri, 1 Jul 2016 13:53:52 +0000 (15:53 +0200)]
initramfs-framework: Add support for PartUUIDs
The rootfs can be addressed also by referring to the PartUUID
value from the GPT.
This patch enables such type of reference.
Signed-off-by: Igor Stoppa <igor.stoppa@intel.com> Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Patrick Ohly [Fri, 1 Jul 2016 13:53:51 +0000 (15:53 +0200)]
initramfs-framework: add retry loop for slow boot devices (like USB)
On some hardware platforms (Gigabyte, qemu), detection of USB devices
by the kernel is slow enough such that it happens only after the first
attempt to mount the rootfs. We need to keep trying for a while
(default: 5s seconds, controlled by roottimeout=<seconds>) and sleep
between each attempt (default: one second, rootdelay=<seconds>).
This change intentionally splits finding the rootfs (in the new
"rootfs") and switching to it ("finish"). That is needed to keep udev
running while waiting for the rootfs, because it shuts down before
"finish" starts. It is also the direction that was discussed on the OE
mailing list for future changes to initramfs-framework (like
supporting a "live CD" module, which would replace or further augment
mounting of the rootfs).
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Fri, 1 Jul 2016 15:58:39 +0000 (16:58 +0100)]
lib/oe/path: remove oe.path.check_output
This was a copy-and-paste of subprocess.check_output() from when we supported
Python <2.7, so simply delete it and use subprocess.check_output() instead.
Signed-off-by: Ross Burton <ross.burton@intel.com>
scripts: Fixed typo in parameter that was causing exception
There is a typo in the logging parameters, "filname" is being used instead of "filename" for yocto-kernel, yocto-layer and wic scripts. This didn't cause issues before since python 2 didn't validate unused parameters but with python >= 3.4.3 an exception is thrown. This patch fixes this parameter name.
[YOCTO #9834]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Khem Raj [Thu, 30 Jun 2016 02:57:56 +0000 (19:57 -0700)]
gcc-5.4: Fix hang with mmusl option on cmdline
When using -m32 -mmusl options in this order, gcc hangs
in parsing the options decode_cmdline_options_to_array()
the reason is that we have broken the link when adding
mmusl options, the order of specifying libc was not kept
in order as a result it was unable to contruct the array
correctly and ended in parse hang.
We fix the options to specify the order properly.
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 29 Jun 2016 03:13:52 +0000 (15:13 +1200)]
classes/populate_sdk_ext: exclude initramfs images from locked signatures
Tasks for image recipes cannot be locked - there's nothing to restore
from shared state to cover them and as a result, if you had "live" in
IMAGE_FSTYPES the build would fail with "taskhash mismatch" errors for
do_rootfs and do_image_complete for the initramfs image recipe, since it
had to try to run those. We should probably catch that issue earlier in
the build and produce a proper error, but for now at least exclude these
signatures from the locked-sigs.inc file so that extensible SDK
installers built when IMAGE_FSTYPES includes "live". (It turned out we
already had code to find other image tasks in the task list in order to
generate the list of install targets.)
Follow-up fix for [YOCTO #9826].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 29 Jun 2016 03:13:51 +0000 (15:13 +1200)]
classes/populate_sdk_ext: ensure tasks to build the image are included
If you build an extensible SDK for an image and IMAGE_FSTYPES includes
"live" then the extensible SDK will fail to install with a bunch of
unexpected task execution errors, matching the missing items required to
build the live image. The issue was we were still depending on do_rootfs
rather than do_image_complete. The fix was slightly more complicated
than just changing the task name as do_image_complete's dependencies are
in the form of dependencies on tasks within the same recipe (represented
in the "deps" varflag rather than the "depends" varflag).
Fixes [YOCTO #9826].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 29 Jun 2016 03:12:03 +0000 (15:12 +1200)]
recipetool: create: avoid decoding errors with Python 3
We're opening source files with the default encoding (utf-8) but we
can't necessarily be sure that they are UTF-8 clean - for example,
recipetool create ftp://mama.indstate.edu/linux/tree/tree-1.7.0.tgz
prior to this patch resulted in a UnicodeDecodeError. Use the
"surrogateescape" mode to avoid this.
Fixes [YOCTO #9822].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
autogen-native: fix script error when processing libguile
do_configure for autogen-native would fail if build directory path
contains '-I' characters, which is caused by the unsuitable sed script
when processing libguile.
Error log:
[snip]
checking for GUILE... yes
awk: fatal: cannot open file `/buildarea1/poky/build /libguile/version.h' for reading (No such file or directory)
configure: error: in `/buildarea1/poky/build-Ice/tmp/work/x86_64-linux/autogen-native/5.18.10-r0/build':
configure: error: cannot determine Guile version
[snip]
Signed-off-by: Zhenbo Gao <zhenbo.gao@windriver.com> Signed-off-by: Yi Zhao <yi.zhao@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Make gettext-native in 0.16.1 recipe provides virtual/gettext-native like
what gettext-native in 0.19.6 recipe does. Otherwise we would fail to start
to do a world build if gettext and gettext-native are set to the low version.
Error message is like below.
ERROR: Multiple versions of gettext-native are due to be built
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Mon, 4 Jul 2016 22:08:13 +0000 (01:08 +0300)]
devshell.bbclass: fix double unbuffering
stdout is already unbuffered in bitbake code. Attempt to
do it again in devshell.bbclass causes this crash when
running devpyshell:
File "scripts/oepydevshell-internal.py", line 29, in <module>
pty = open(sys.argv[1], "w+b", 0)
IOError: [Errno 13] Permission denied: '/dev/pts/6'
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:03 +0000 (00:07 +1200)]
classes/populate_sdk_ext: eliminate double execution on install
Use the new BB_SETSCENE_ENFORCE functionality to avoid having to run
bitbake twice on installing the extensible SDK - we can now do it all in
one invocation which not only takes less time, but we should also get
more meaningful errors for some types of failure, in particular where
downloading from an sstate mirror fails.
One result of this change is that you get the errors printed on the
console during normal output rather than this going to the
preparing_build_system.log file first. In OE-Core revision 227d2cbf9e0b8c35fa6644e3d72e0699db9607fa, we changed to always print the
contents of preparing_build_system.log on failure, but now at least the
error contents of that log is duplicated. Besides, I intentionally
didn't print out the contents of that log during normal usage because
it's quite verbose - the bug that we were attempting to fix was about
not getting this information when seeing failures in the automated
tests, thus I've moved printing the log to the test handling code
instead.
Part of the implementation of [YOCTO #9367].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:06 +0000 (00:07 +1200)]
toolchain-shar-extract.sh: allow TERM through into SDK installer env
In order to show the task progress during the extensible SDK
installation, knotty needs to know it is using a terminal it can
support the "interactive" mode on (i.e. ncurses can be used), which
means it needs access to the TERM variable value, so allow that
through into the cleaned environment within the SDK installation
script.
Part of the implementation for [YOCTO #9613].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:04 +0000 (00:07 +1200)]
lib/oe/sstatesig: print locked sigs file message only when explicitly called
If we're building the extensible SDK we don't need to see the "Writing
locked sigs" message; it's only necessary when the user explicitly runs
bitbake -S none <target>.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:02 +0000 (00:07 +1200)]
classes/testsdk: print output correctly on failure
With Python 3 we get a bytes object from the command output and not a
string, which gives some ugly formatting for error messages unless you
decode it first.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:01 +0000 (00:07 +1200)]
classes/sstate: add a mode to error if sstate package unavailable
If BB_SETSCENE_ENFORCE is set to "1" and an sstate package fails to
download outside of the whitelist specified by
BB_SETSCENE_ENFORCE_WHITELIST, then fail immediately so you can tell
that the problem was caused by failing to restore the task from sstate.
Part of the implementation of [YOCTO #9367].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:07:00 +0000 (00:07 +1200)]
classes/sstate: show progress during sstate object availability check
Show progress through bitbake's standard terminal UI when checking for
shared state object availability, since this can take some time if there
are a large number of tasks to be executed and/or the network connection
is slow.
Part of the implementation for [YOCTO #5853].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Thu, 23 Jun 2016 12:06:59 +0000 (00:06 +1200)]
classes/image: implement progress support for do_rootfs
Use the new task progress functionality to report progress during
do_rootfs. This is a little coarse and ideally we would have some
progress within the installation section, but it's better than
nothing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 29 Jun 2016 03:09:16 +0000 (15:09 +1200)]
local.conf.sample.extended: drop RPM4 example
We don't have RPM4 support anymore, so there's no point including this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
