Ed Bartosh [Tue, 31 Jan 2017 09:46:11 +0000 (11:46 +0200)]
wic: renamd direct_plugin.py -> direct.py
As this files is located in plugins/imager subdirectory it's
obvious that it's an imager plugin. Renamed to direct.py to
be consistent with plugin naming scheme.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Tue, 31 Jan 2017 01:43:56 +0000 (03:43 +0200)]
wic: improve naming in direct_plugin classes
Synchronized attribure names in DirectImageCreator and
DirectPlugin for better readability. Simplified code,
removed unneeded global variable disk_methods.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Mon, 30 Jan 2017 21:54:29 +0000 (23:54 +0200)]
wic: moved content of direct.py to direct_plugin
This move simplifies directory structure and makes
further refactoring easier. The code from direct.py was used
only in direct_plugin, so it's safe to move it there.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Mon, 30 Jan 2017 21:31:53 +0000 (23:31 +0200)]
wic: direct_plugin: stop using config manager
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
Got rid of using configmgr global object in direct_plugin and direct
modules. It was used to implicitly parse kickstart file and set
couple of variables.
Replaced usage of configmgr by passing parameters directly to the
DirectImageCreator.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Mon, 30 Jan 2017 21:23:00 +0000 (23:23 +0200)]
wic: creator: stop using config manager
This is a preparation to removing conf.py and config/wic.conf
from the codebase.
confmgr object is complicated for no reason and almost
useless as all configuration info comes from command line and
bitbake variables. It's used it creator.py to store information
about output directory, logs and some never used functionality
like tmpfs for future use, which doesn't actually happen.
[YOCTO #10619]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ross Burton [Tue, 31 Jan 2017 20:24:32 +0000 (20:24 +0000)]
pkg-config: depend on pkgconfig-native for target builds
When building for the target, pkg-config uses the target glib-2.0 instead of
it's own minimal fork. To find this it needs to use pkg-config so ensure this
dependency exists in case it doesn't exist on the host already.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Richard Purdie [Wed, 1 Feb 2017 11:29:06 +0000 (11:29 +0000)]
staging: Reduce the number of mkdirs calls
The number of mkdir calls was showing up high on the profile charts since
it was getting called once per file which is excessive. Each call results
in one or more syscalls which is bad for performance. Cache which
directories we've seen to reduce the calls to a more reasonable number
and speed up recipe specific sysroots.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 1 Feb 2017 11:25:32 +0000 (11:25 +0000)]
sstate/staging: Batch log messages for performance
According to profile data, repeated calls to bb.debug and bb.note in
the extend_recipe_sysroot() codepath were accounting for 75% of the time
(1.5s) in calls from tasks like do_image_complete.
This batches up the log messages into one call into the logging system
which gives similar behaviour to disabling the logging but retains the
debug information.
Since setscene_depvalid is also called from bitbake's setscene code,
we have to be a little creative with the function parameters and leave
the other debug output mechanism in place. This should hopefully
speed up recipe specific sysroots.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 30 Jan 2017 23:18:09 +0000 (23:18 +0000)]
binutils-cross: Remove exec_prefix from the linker search path
We don't put target libs into a native/cross ${exec_prefix} but having
this in the default search path means all linker scripts have to be relocated.
This is a considerable chunk of files to create multiple copies of for no good
reason.
Instead, patch out the paths we don't need.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 30 Jan 2017 23:11:32 +0000 (23:11 +0000)]
distro/defaultsetup: Enable removal of libtool .la files by default
Relocation of native .la files during recipe specific sysroot relocation
is probably the final straw in just killing these files off.
Change things so this class is inherited by default. If distros don't want to
do this, they can opt out but it seems like the best thing to do now since
.la files aren't needed on Linux.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 30 Jan 2017 23:08:38 +0000 (23:08 +0000)]
relocatable: Make native .pc files relocatable
The native .pc files currently have hardcoded paths in them meaning each has
to be relocated at final install time. pkg-config has built in functionality
to avoid this, namely the pcfiledir variable.
This function translates .pc files to use the variable meaning further
relocation later is unnecessary.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 30 Jan 2017 23:08:11 +0000 (23:08 +0000)]
utils: Avoid need to relocate wrapper scripts
Whilst the path to the executable is dynamically determined, the passed in
environment variables or parameters are not relocatable and rely on the sstate
remapping code. In the recipe specific sysroot case this has become more costly.
This patch translates such paths into relocatable entries which means that
a sed replacement at final installation isn't needed.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:33 +0000 (13:50 +0100)]
verify-bashisms: support warnings with more than one line of source code
All warnings start with "possible bashism in", followed by one or more
(in the case of line continuation) lines of source code. To support
more than one line, we now split by matching against the known intro
text.
Example:
$ verify-bashisms guile
...
/.../openembedded-core/meta/recipes-devtools/guile/guile_2.0.13.bb
possible bashism in guile_cross_config line 94 ($'...' should be "$(printf '...')"):
echo '#!'`which ${BUILD_SYS}-guile`$' \\\n--no-auto-compile -e main -s\n!#\n(define %guile-build-info '\'\( \
> ${B}/guile-config.cross
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:31 +0000 (13:50 +0100)]
verify-bashisms: check scripts only once, include original file and line
Several scripts that are defined in .bbclass files end up in multiple
different recipes. It's better (faster, less repetitive error reports)
to check them only once.
In addition, the real information for the developer is where he can
find the script, not which recipe file uses it. verify-bashisms now
prints the original file instead of the recipe whenever possible
(i.e. 'filename' is set) and also bumps the line number so that it is
relative to the file and not the script.
Example with one real error and one added just for testing:
/.../openembedded-core/meta/classes/populate_sdk_ext.bbclass
possible bashism in install_tools line 515 (should be 'b = a'):
if [ "${SDK_INCLUDE_TOOLCHAIN}" == "1" -a ! -e $unfsd_path ] ; then
possible bashism in install_tools line 521 (type):
type fixme
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:29 +0000 (13:50 +0100)]
verify-bashisms: fix problems with tinfoil2
tinfoil2 is based on a client/server architecture, which broke the
verify-bashisms script:
- The tinfoil instance and its data proxies can't be pickled, so
all interaction with the bitbake server has to run in the main
script process and only processing of the plain scripts can
be done with multiprocessing:
- The multiprocessing pool has to be created before initializing
tinfoil, otherwise the pool workers end up trying to communicate
with the bitbake server during shutdown:
ERROR: UI received SIGTERM
Process ForkPoolWorker-2:
Traceback (most recent call last):
File "/usr/lib/python3.4/multiprocessing/process.py", line 257, in _bootstrap
util._exit_function()
File "/usr/lib/python3.4/multiprocessing/util.py", line 286, in _exit_function
_run_finalizers(0)
...
File "/usr/lib/python3.4/multiprocessing/process.py", line 131, in is_alive
assert self._parent_pid == os.getpid(), 'can only test a child process'
AssertionError: can only test a child process
- func() needs to defined before creating the pool to avoid:
AttributeError: Can't get attribute 'func' on <module '__main__' from '/work/openembedded-core/scripts/verify-bashisms'>
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Tue, 31 Jan 2017 12:50:27 +0000 (13:50 +0100)]
verify-bashisms: point out where to get checkbashisms.pl
The current SourceForge project seems to be unmaintained (last release
2.0.0.2 from 2015) while the copy used by Debian is quite active (last
commit 2016-09-30).
Ideally, checkbashisms.pl should get installed automatically via a
recipe, but for now at least provide the link for manual installation.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 31 Jan 2017 13:23:30 +0000 (15:23 +0200)]
at-spi2-atk: depend on glib-2.0-native
Missing dependency revealed by recipe specific sysroots. If the host
system does not have glib-compile-schemas installed have (via
libglib2.0-dev, glib2, glib2-tools or similar) do_configure would fail
with:
configure: error: glib-compile-schemas not found.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Tue, 31 Jan 2017 13:23:28 +0000 (15:23 +0200)]
rpm: inherit pkgconfig
Rpm needs pkg-config in order to build successfully. Recipe specific
sysroots revealed this missing dependency when trying to build on a host
without pkg-config.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
core-image-tiny-initramfs: Fix error message shown after a successful initrd boot
When booting core-image-tiny-initramfs, since we want to live on initrd,
on purpose, we never find a rootfs image to switch root to,
this causes init to show an error as it would with other images,
this patch replaces the message shown to the user, avoiding confusion
when it was indeed a successful boot.
Yannick Gicquel [Wed, 25 Jan 2017 20:19:46 +0000 (21:19 +0100)]
ccache: update to 3.3.3
Release note available at this location:
https://ccache.samba.org/releasenotes.html#_ccache_3_3_3
Moving to tarball source as git.samba.org is not used anymore.
License checksum change is due to an update in URL, now refering to "https"
protocol instead of "http", line 32:
'https://ccache.samba.org/authors.html'
Signed-off-by: Yannick Gicquel <yannick.gicquel@iot.bzh> Signed-off-by: Ross Burton <ross.burton@intel.com>
Juro Bystricky [Wed, 25 Jan 2017 20:54:57 +0000 (12:54 -0800)]
runqemu: allow bypassing of network setup
At present it is silently assumed all QEMU machines support networking.
As a consequence, one cannot run QEMUs without network emulation
using "runqemu".
This patch allows bypassing any network setup providing the qemuboot.conf
file contains:
qb_net = none
[YOCTO#10661]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mariano Lopez [Wed, 25 Jan 2017 12:20:06 +0000 (12:20 +0000)]
testimage.bbclass: Allow to run tests on autobuilder's images
With the change to the new framework data store dependecy was
removed, instead a new file is generated and used in testimage.
When testing builds from the autobuilders the test data values
are from the autobuilder, including the paths.
Some tests require paths to current environment in order to run,
this commit will update such paths and fix the error of running
images donwloaded from autobuilders.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mariano Lopez [Wed, 25 Jan 2017 12:20:05 +0000 (12:20 +0000)]
runtime/cases/smart.py: Check for IMAGE_PKGTYPE instead of PACKAGE_CLASSES
smart test requires to build the image using rpm packages, this check was
included, but it checked for PACKAGE_CLASSES=='package_rpm', and this is
not true when building packages for rpm and deb/ipk. So this would check
IMAGE_PKGTYPE instead.
[YOCTO #10964]
Signed-off-by: Mariano Lopez <mariano.lopez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Updates sanity tests to cope with minimal eSDK installer
1. Skips the validation of sanity if packagegroup-cross-canadian is in
host package.
2. Skips if SDK does not include toolchain at cases/devtool.py
This should fix [YOCTO #10794]
Signed-off-by: Francisco Pedraza <francisco.j.pedraza.gonzalez@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Kristian Amlie [Wed, 25 Jan 2017 14:46:15 +0000 (15:46 +0100)]
sstate: Make SSTATE_SCAN_CMD vars configurable using weak defaults.
For this we move them out of the python section so they can be
overridden on a per-recipe basis.
The motivation for this change is that not all tool chains need the
path modifications provided by the command, and these will provide
alternative or empty commands. The Go compiler is such an example.
Signed-off-by: Kristian Amlie <kristian.amlie@mender.io> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Wed, 25 Jan 2017 13:18:36 +0000 (15:18 +0200)]
toolchain-shar-extract: compare SDK and host gcc versions
If ext sdk is built by gcc version higher than host gcc version
and host gcc version is 4.8 or 4.9 the installation is known to
fail due to the way uninative sstate package is built.
It's a known issue and we don't have a way to fix it for above
mentioned combinations of build and host gcc versions.
Detected non-installable combinations of gcc versions and
print an installation error.
[YOCTO #10881]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ismo Puustinen [Wed, 25 Jan 2017 13:03:35 +0000 (15:03 +0200)]
bluez5: make readline support conditional.
Add readline support to PACKAGECONFIG. If readline needs if left out
of compilation, the bluez utilities which depend on readline are not
included in build or packages.
The defaults in PACKAGECONFIG are the same as before, so there should be
no change to current users.
Signed-off-by: Ismo Puustinen <ismo.puustinen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Thu, 26 Jan 2017 23:28:05 +0000 (23:28 +0000)]
default-distrovars: add acl to DISTRO_FEATURES
ACLs are pretty useful in the modern world, with security systems such as
SELinux and Smack that use them extensively. As the overhead is minimal, add
ACL to DEFAULT_DISTRO_FEATURES so that support for them is enabled by default.
The overhead for core-image-sato is that coreutils, libarchive, and opkg link
against libacl. The size increase of those packages is minimal, and libacl is
35kb.
[ YOCTO #8200 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 25 Jan 2017 17:13:40 +0000 (17:13 +0000)]
package_deb: handle allarch when mapping tune to dpkg architectures
It was discovered that buildtools-tarball can't be built with Debian packaging:
E: Unable to locate package nativesdk-buildtools-perl-dummy
The package exists but dpkg doesn't see it because the Architecture field isn't
valid: for this package the Architecture field is 'allarch' which isn't in the
list of valid architectures we define.
package_deb already has a mapping of OE architectures (PACKAGE_ARCH) to dpkg
architectures (DPKG_ARCH), for example our x86_64 is amd64 in dpkg. However
allarch.bbclass sets an invalid TARGET_ARCH of 'allarch' (to break anything
which attempts to use the target architecture) and package_deb relied on the
PACKAGE_ARCH check later turning the Architecture field into 'all'.
This usually works, but nativesdk-buildtools-perl-dummy inherits allarch but
then changes PACKAGE_ARCH to buildtools-dummy-nativesdk to isolate the packages
in a separate feed. We can handle this by explictly checking for allarch in the
TARGET_ARCH check, as that everything that inherits allarch will want to have
Architecture: all in their packages.
[ YOCTO #10700 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Wed, 25 Jan 2017 16:21:08 +0000 (16:21 +0000)]
distrodata: add DATETIME to vardepsexclude for do_distro_check.
This task uses DATETIME so add it to vardepsexclude so bitbake doesn't refuse to
execute the task when the time changes between the controller and the worker
calculating the hash.
[ YOCTO #10960 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Mon, 16 Jan 2017 12:23:08 +0000 (14:23 +0200)]
isoimage-isohybrid: use TRANSLATED_TARGET_ARCH instead of MACHINE_ARCH
isoimage-sihybrid plugin uses MACHINE_ARCH to get the name of initrd image.
It doesn't work for all machines, for example for quemux86-64 machine
MACHINE_ARCH is quemux86_64 and initrd name is
core-image-minimal-initramfs-qemux86-64.cpio.gz
Used TRANSLATED_TARGET_ARCH variable to get the initrd image name.
Replaced MACHINE_ARCH->TRANSLATED_TARGET_ARCH in WICVARS variable to
make it available from <image>.env file.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Tom Zanussi [Tue, 12 Jul 2016 21:50:03 +0000 (16:50 -0500)]
wic: Look for image artifacts in a common location
Rather than have each image type look for artifacts in image-specific
locations, move towards having them look for artifacts in a common
location, in this case DEPLOY_DIR_IMAGE
Use the existing deploy.bbclass to have the bootloaders put their binaries
in DEPLOY_DIR_IMAGE and then wic will find them and place them in the image
Ed Bartosh [Thu, 12 Jan 2017 17:00:10 +0000 (19:00 +0200)]
wic: isoimage-isohybrid: stop using HDDDIR
Stop using HDDDIR in isoimage-isohybrid wic plugin.
This variable is set by hddimg code, which is going to be
removed soon.
All required artifacts should be available from ISODIR.
wic-image.bbclass has been modified to build iso artifacts,
so it should be safe to remove usage of HDDDIR.
[YOCTO #10835]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Thu, 26 Jan 2017 11:52:24 +0000 (13:52 +0200)]
image-wic: move wic code to image-wic.bbclass
There is a lot of wic code in image.bbclass and image_types.bbclass
Having all code separated in one file should make it more readable
and easier to maintain.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Tue, 24 Jan 2017 13:43:56 +0000 (15:43 +0200)]
selftest: wic: test default output directory
As tests now explicitly specify output directory we don't
need test_alternate_output_dir test case. However, we need
to test wic output to default output location.
Removed test_alternate_output_dir test case.
Added test_default_output_dir test case.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
wic started to use current directory as a default output dir.
Specified output directory in wic command line to make tests
more predictable and easier to maintain.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:54 +0000 (15:43 +0200)]
image_types: use correct output directory
Wic put result images into <output dir>/build, which was confusing.
Now it's fixed in wic code and images are put into output directory.
Changed code in image_types to reflect this.
[YOCTO #10783]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:53 +0000 (15:43 +0200)]
wic: direct.py: get rid of names with two underscores
Attributes with two leading underscores are mangled in Python
and used mainly for avoiding name clashes with names from
subclasses. They're not needed in most of wic classes.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 24 Jan 2017 13:43:50 +0000 (15:43 +0200)]
wic: get rid of baseimager inheritance
Simplified DirectImageCreator code by removing inheritance
from BaseImageCreator. This inheritance doesn't make much sense
as DirectImageCreator is the only class that was inherited from
BaseImageCreator.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 28 Jan 2017 14:40:36 +0000 (14:40 +0000)]
staging: Handle stale sysroot rebuild issue
If a stale sysroot object was found the complete stamp wasn't removed,
meaning later code could fail when the new link creation was attempted.
Ensure the stale complete stamp is also removed if present.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Thu, 26 Jan 2017 11:28:17 +0000 (13:28 +0200)]
selftest: wic: split test_debug test case
Splitted to test_debug_short and test_debug_long to make each
of the test cases to run wic once. This is consistent with the
rest of the test cases and ensures that test cases are set up
properly.
This also fixes the following test failure caused by the image
left from the first wic run:
FAIL: test_debug (oeqa.selftest.wic.Wic)
Test debug
----------------------------------------------------------------------
Traceback (most recent call last):
File "meta/lib/oeqa/utils/decorators.py", line 109, in wrapped_f
return func(*args, **kwargs)
File "meta/lib/oeqa/selftest/wic.py", line 270, in test_debug
self.assertEqual(1, len(glob(self.resultdir + "directdisk-*.direct")))
AssertionError: 1 != 2
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
