Patrick Ohly [Tue, 31 Jan 2017 12:50:33 +0000 (13:50 +0100)]
verify-bashisms: support warnings with more than one line of source code
All warnings start with "possible bashism in", followed by one or more
(in the case of line continuation) lines of source code. To support
more than one line, we now split by matching against the known intro
text.
Example:
$ verify-bashisms guile
...
/.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb
possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"):
echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \
> ${B}/guile-config.cross
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:31 +0000 (13:50 +0100)]
verify-bashisms: check scripts only once, include original file and line
Several scripts that are defined in .bbclass files end up in multiple
different recipes. It's better (faster, less repetitive error reports)
to check them only once.
In addition, the real information for the developer is where he can
find the script, not which recipe file uses it. verify-bashisms now
prints the original file instead of the recipe whenever possible
(i.e. 'filename' is set) and also bumps the line number so that it is
relative to the file and not the script.
Example with one real error and one added just for testing:
/.../openembedded-core/meta/classes/populate_sdk_ext.bbclass
possible bashism in install_tools line 515 (should be 'b = a'):
if [ "${SDK_INCLUDE_TOOLCHAIN}" == "1" -a ! -e $unfsd_path ] ; then
possible bashism in install_tools line 521 (type):
type fixme
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:29 +0000 (13:50 +0100)]
verify-bashisms: fix problems with tinfoil2
tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:
- The tinfoil instance and its data proxies can't be pickled, so
all interaction with the bitbake server has to run in the main
script process and only processing of the plain scripts can
be done with multiprocessing:
- The multiprocessing pool has to be created before initializing
tinfoil, otherwise the pool workers end up trying to communicate
with the bitbake server during shutdown:
ERROR: UI received SIGTERM
Process ForkPoolWorker-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
util._exit_function()
File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
_run_finalizers(0)
...
File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
- func() needs to defined before creating the pool to avoid:
AttributeError: Can't get attribute 'func' on <module '__main__' from '/work/openembedded-core/scripts/verify-bashisms'>
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:27 +0000 (13:50 +0100)]
verify-bashisms: point out where to get checkbashisms.pl
The current SourceForge project seems to be unmaintained (last release
2.0.0.2 from 2015) while the copy used by Debian is quite active (last
commit 2016-09-30).
Ideally, checkbashisms.pl should get installed automatically via a
recipe, but for now at least provide the link for manual installation.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 31 Jan 2017 13:23:30 +0000 (15:23 +0200)]
at-spi2-atk: depend on glib-2.0-native
Missing dependency revealed by recipe specific sysroots. If the host
system does not have glib-compile-schemas installed have (via
libglib2.0-dev, glib2, glib2-tools or similar) do_configure would fail
with:
configure: error: glib-compile-schemas not found.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 31 Jan 2017 13:23:28 +0000 (15:23 +0200)]
rpm: inherit pkgconfig
Rpm needs pkg-config in order to build successfully. Recipe specific
sysroots revealed this missing dependency when trying to build on a host
without pkg-config.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
core-image-tiny-initramfs: Fix error message shown after a successful initrd boot
When booting core-image-tiny-initramfs, since we want to live on initrd,
on purpose, we never find a rootfs image to switch root to,
this causes init to show an error as it would with other images,
this patch replaces the message shown to the user, avoiding confusion
when it was indeed a successful boot.
Yannick Gicquel [Wed, 25 Jan 2017 20:19:46 +0000 (21:19 +0100)]
ccache: update to 3.3.3
Release note available at this location:
https://ccache.samba.org/releasenotes.html#_ccache_3_3_3
Moving to tarball source as git.samba.org is not used anymore.
License checksum change is due to an update in URL, now refering to "https"
protocol instead of "http", line 32:
'https://ccache.samba.org/authors.html'
Signed-off-by: Yannick Gicquel <yannick.gicquel@iot.bzh> Signed-off-by: Ross Burton <ross.burton@intel.com>
Juro Bystricky [Wed, 25 Jan 2017 20:54:57 +0000 (12:54 -0800)]
runqemu: allow bypassing of network setup
At present it is silently assumed all QEMU machines support networking.
As a consequence, one cannot run QEMUs without network emulation
using "runqemu".
This patch allows bypassing any network setup providing the qemuboot.conf
file contains:
qb_net = none
[YOCTO#10661]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mariano Lopez [Wed, 25 Jan 2017 12:20:06 +0000 (12:20 +0000)]
testimage.bbclass: Allow to run tests on autobuilder's images
With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.
Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mariano Lopez [Wed, 25 Jan 2017 12:20:05 +0000 (12:20 +0000)]
runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES
smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Updates sanity tests to cope with minimal eSDK installer
1. Skips the validation of sanity if packagegroup-cross-canadian is in
host package.
2. Skips if SDK does not include toolchain at cases/devtool.py
This should fix [YOCTO #10794]
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Kristian Amlie [Wed, 25 Jan 2017 14:46:15 +0000 (15:46 +0100)]
sstate: Make SSTATE_SCAN_CMD vars configurable using weak defaults.
For this we move them out of the python section so they can be
overridden on a per-recipe basis.
The motivation for this change is that not all tool chains need the
path modifications provided by the command, and these will provide
alternative or empty commands. The Go compiler is such an example.
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Wed, 25 Jan 2017 13:18:36 +0000 (15:18 +0200)]
toolchain-shar-extract: compare SDK and host gcc versions
If ext sdk is built by gcc version higher than host gcc version
and host gcc version is 4.8 or 4.9 the installation is known to
fail due to the way uninative sstate package is built.
It's a known issue and we don't have a way to fix it for above
mentioned combinations of build and host gcc versions.
Detected non-installable combinations of gcc versions and
print an installation error.
[YOCTO #10881]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ismo Puustinen [Wed, 25 Jan 2017 13:03:35 +0000 (15:03 +0200)]
bluez5: make readline support conditional.
Add readline support to PACKAGECONFIG. If readline needs if left out
of compilation, the bluez utilities which depend on readline are not
included in build or packages.
The defaults in PACKAGECONFIG are the same as before, so there should be
no change to current users.
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Thu, 26 Jan 2017 23:28:05 +0000 (23:28 +0000)]
default-distrovars: add acl to DISTRO_FEATURES
ACLs are pretty useful in the modern world, with security systems such as
SELinux and Smack that use them extensively. As the overhead is minimal, add
ACL to DEFAULT_DISTRO_FEATURES so that support for them is enabled by default.
The overhead for core-image-sato is that coreutils, libarchive, and opkg link
against libacl. The size increase of those packages is minimal, and libacl is
35kb.
[ YOCTO #8200 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 25 Jan 2017 17:13:40 +0000 (17:13 +0000)]
package_deb: handle allarch when mapping tune to dpkg architectures
It was discovered that buildtools-tarball can't be built with Debian packaging:
E: Unable to locate package nativesdk-buildtools-perl-dummy
The package exists but dpkg doesn't see it because the Architecture field isn't
valid: for this package the Architecture field is 'allarch' which isn't in the
list of valid architectures we define.
package_deb already has a mapping of OE architectures (PACKAGE_ARCH) to dpkg
architectures (DPKG_ARCH), for example our x86_64 is amd64 in dpkg. However
allarch.bbclass sets an invalid TARGET_ARCH of 'allarch' (to break anything
which attempts to use the target architecture) and package_deb relied on the
PACKAGE_ARCH check later turning the Architecture field into 'all'.
This usually works, but nativesdk-buildtools-perl-dummy inherits allarch but
then changes PACKAGE_ARCH to buildtools-dummy-nativesdk to isolate the packages
in a separate feed. We can handle this by explictly checking for allarch in the
TARGET_ARCH check, as that everything that inherits allarch will want to have
Architecture: all in their packages.
[ YOCTO #10700 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 25 Jan 2017 16:21:08 +0000 (16:21 +0000)]
distrodata: add DATETIME to vardepsexclude for do_distro_check.
This task uses DATETIME so add it to vardepsexclude so bitbake doesn't refuse to
execute the task when the time changes between the controller and the worker
calculating the hash.
[ YOCTO #10960 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 16 Jan 2017 12:23:08 +0000 (14:23 +0200)]
isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCH
isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image.
It doesn't work for all machines, for example for quemux86-64 machine
MACHINE_ARCH is quemux86_64 and initrd name is
core-image-minimal-initramfs-qemux86-64.cpio.gz
Used TRANSLATED_TARGET_ARCH variable to get the initrd image name.
Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to
make it available from <image>.env file.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Tom Zanussi [Tue, 12 Jul 2016 21:50:03 +0000 (16:50 -0500)]
wic: Look for image artifacts in a common location
Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE
Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image
Ed Bartosh [Thu, 12 Jan 2017 17:00:10 +0000 (19:00 +0200)]
wic: isoimage-isohybrid: stop using HDDDIR
Stop using HDDDIR in isoimage-isohybrid wic plugin.
This variable is set by hddimg code, which is going to be
removed soon.
All required artifacts should be available from ISODIR.
wic-image.bbclass has been modified to build iso artifacts,
so it should be safe to remove usage of HDDDIR.
[YOCTO #10835]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Thu, 26 Jan 2017 11:52:24 +0000 (13:52 +0200)]
image-wic: move wic code to image-wic.bbclass
There is a lot of wic code in image.bbclass and image_types.bbclass
Having all code separated in one file should make it more readable
and easier to maintain.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Tue, 24 Jan 2017 13:43:56 +0000 (15:43 +0200)]
selftest: wic: test default output directory
As tests now explicitly specify output directory we don't
need test_alternate_output_dir test case. However, we need
to test wic output to default output location.
Removed test_alternate_output_dir test case.
Added test_default_output_dir test case.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic started to use current directory as a default output dir.
Specified output directory in wic command line to make tests
more predictable and easier to maintain.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:54 +0000 (15:43 +0200)]
image_types: use correct output directory
Wic put result images into <output dir>/build, which was confusing.
Now it's fixed in wic code and images are put into output directory.
Changed code in image_types to reflect this.
[YOCTO #10783]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:53 +0000 (15:43 +0200)]
wic: direct.py: get rid of names with two underscores
Attributes with two leading underscores are mangled in Python
and used mainly for avoiding name clashes with names from
subclasses. They're not needed in most of wic classes.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:50 +0000 (15:43 +0200)]
wic: get rid of baseimager inheritance
Simplified DirectImageCreator code by removing inheritance
from BaseImageCreator. This inheritance doesn't make much sense
as DirectImageCreator is the only class that was inherited from
BaseImageCreator.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 28 Jan 2017 14:40:36 +0000 (14:40 +0000)]
staging: Handle stale sysroot rebuild issue
If a stale sysroot object was found the complete stamp wasn't removed,
meaning later code could fail when the new link creation was attempted.
Ensure the stale complete stamp is also removed if present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 26 Jan 2017 11:28:17 +0000 (13:28 +0200)]
selftest: wic: split test_debug test case
Splitted to test_debug_short and test_debug_long to make each
of the test cases to run wic once. This is consistent with the
rest of the test cases and ensures that test cases are set up
properly.
This also fixes the following test failure caused by the image
left from the first wic run:
FAIL: test_debug (oeqa.selftest.wic.Wic)
Test debug
----------------------------------------------------------------------
Traceback (most recent call last):
File "meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
return func(*args, **kwargs)
File "meta/lib/oeqa/selftest/wic.py", line 270, in test_debug
self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
AssertionError: 1 != 2
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 27 Jan 2017 16:55:24 +0000 (16:55 +0000)]
staging: Allow removal of stale sysroot objects
The main sysroot components of unreachable build targets will be removed by the core
code. This currently doesn't trigger a removal in the individual workdirs. This
adds in symlinking between the complete stamps and the component sysroot meaning
we can detect when someting was removed and hence remove it from the sysroot.
This fixes cases where DISTRO_FEATURES like systemd are changed amongst other
things and makes builds more robust against configuration changes.
If a dependency is rebuild, that is caught by checksum comparision code elsewhere
in this function as before.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 27 Jan 2017 16:53:13 +0000 (16:53 +0000)]
staging: Drop common prefix WORKDIR from manifest files
Manifest files containing the same duplicated prefix are wasteful on space
and ultimately this costs build time. Drop the WORKDIR prefix from the manifest
files since this small change mounts up a lot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 27 Jan 2017 16:50:57 +0000 (16:50 +0000)]
sstate: Allow sstate_clean_manifest to take a prefix
Manifest files containing the same duplicated prefix are wasteful on space
and ultimately this costs build time. Add support for manifest files with
common prefixes removed and use the prefix if the path isn't absolute.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 27 Jan 2017 13:32:06 +0000 (13:32 +0000)]
allarch: Drop STAGING_DIR_HOST expansion
Now that STAGINGDIR_HOST doesn't contain MACHINE, we no longer need to expand
the value. Pre-expansion can mean components like PV can be expanded too early
and cause problems for certain use cases.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
We used to have issues removing tasks like do_fetch due to implications
for targets like world and universe. These have now been resolved.
Removing uneeded tasks has advantages compared to noexec since it means
that accidentally left in dependencies are no longer needed/processed
(e.g. do_patch depends on quilt-native).
This cleans up a number of cases which local analysis highlighted as
being unneeded leading to slightly cleaner task graphs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Wed, 25 Jan 2017 11:07:48 +0000 (12:07 +0100)]
image_types.bbclass: rebuild .wks file when .wks.in changes
WKS_FILE(S) can refer to .wks.in files which get expanded during the
build by do_write_wks_template. The actual content of the .wks.in file
gets added to the recipe meta data during parsing, and thus we need to
ensure that the recipe gets re-parsed when the file changes.
This fixes two related problems:
- editing the .wks.in file and rebuilding an image did not recreate
the image unless something else changed or "bitbake -c clean" was
used explicitly
- when forcing a rebuild, the cached meta data and the actual one
do not match, leading to "ERROR: Taskhash mismatch ... for ....bb.do_write_wks_template"
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Maxin B. John [Mon, 23 Jan 2017 09:59:00 +0000 (11:59 +0200)]
libpng12: Upgrade 1.2.56 -> 1.2.57
This upgrade fixes the vulnerability: CVE-2016-10087
License file changes are due to updates in Version and Copyright date. ie:
'libpng versions 1.0.7, July 1, 2000, through 1.2.57, December 29, 2016, are
Copyright (c) 2000-2002, 2004, 2006-2016 Glenn Randers-Pehrson'
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
