Markus Lehtonen [Mon, 25 Jan 2016 12:21:34 +0000 (14:21 +0200)]
meta/lib: new module for handling GPG signing
Add a new Python module (oe.gpg_sign) for handling GPG signing
operations, i.e. currently package and package feed signing. The purpose
is to be able to more easily support various signing backends and to be
able to centralise signing functionality into one place (e.g. package
signing and sstate signing). Currently, only local signing with gpg is
implemented.
[YOCTO #8755]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Tue, 26 Jan 2016 02:53:54 +0000 (15:53 +1300)]
devtool: add configure-help subcommand
When you need to set EXTRA_OECONF for a recipe, you need to know what
options the configure script actually supports; the configure script
however is only accessible from within a devshell and (at least in the
case of autotooled software fetched from an SCM repository) may not
actually exist until do_configure has run. Thus, provide a "devtool
configure-help" subcommand that runs the configure script for a recipe
with --help and shows you the output through a pager (e.g. less),
prefaced by a header describing the current options being specified.
There is basic support for autotools, cmake and bare configure scripts.
The cmake support is a little hacky since cmake doesn't really have a
concise help option that lists user-defined knobs (without actually
running through the configure process), however that being a design
feature of cmake there's not much I can think of to do about that at
the moment.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 26 Jan 2016 02:53:57 +0000 (15:53 +1300)]
devtool: properly handle bb.build.FuncFailed when extracting source
When we run the tasks required to extract the source for a recipe (e.g.
within "devtool modify" or "devtool extract") if one of those tasks
fails you get a bb.build.FuncFailed exception; handle this properly so
you don't see a traceback.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 26 Jan 2016 02:53:56 +0000 (15:53 +1300)]
devtool: add: warn if modified recipe found in attic directory
If a recipe generated by "devtool add" has been modified since then when
you run "devtool reset", it will be moved into the "attic" subdirectory
of the workspace in case those modifications need to be preserved. It
seems natural that if those modifications were worth preserving we
should warn the user if such a file exists when they run "devtool add"
to create the same recipe again, so they can pick up where they left off
if they want to.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 26 Jan 2016 02:53:55 +0000 (15:53 +1300)]
devtool: build-image: allow specifying packages to add to image
Provide an option to devtool build-image to specify the list of packages
instead of taking the list of packages produced by recipes in the
workspace. Sometimes you don't want all of these packages; other times
you want to add more.
This is the most immediate fix for [YOCTO #8855], though it is a little
crude so I would like to provide better means of customising the image
contents later.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 26 Jan 2016 18:02:21 +0000 (18:02 +0000)]
image: Don't create tasks with '.' in the name
Similarly to "-", "." doesn't work well in task names but is used in
some real world image classes. Work around this with some replacements
for now to unbreak layers.
(Issues don't show themselves until runtime, e.g. with --dry-run)
Tested-By: Otavio Salvador <otavio.salvador@ossystems.com.br> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 25 Jan 2016 15:07:34 +0000 (15:07 +0000)]
kernel: Clean DEPLOYDIR before do_deploy runs
If we don't do this, the deploy sstate object contains an every
increasing number of modules tarballs and kernel images, one per
execution of "-c deploy -f".
Cleaning the directory before we start makes things much tidier.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 25 Jan 2016 15:05:18 +0000 (15:05 +0000)]
libtirpc: Drop unneeded xz-native dependency
xz-native is needed to decompress the sources but DEPENDS isn't correct
for this, its needed at do_unpack time. base.bbclass already handles
this correctly so drop the unneeded dependency.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 25 Jan 2016 15:04:47 +0000 (15:04 +0000)]
libuser: Drop unneeded xz-native dependency
xz-native is needed to decompress the sources but DEPENDS isn't correct
for this, its needed at do_unpack time. base.bbclass already handles
this correctly so drop the unneeded dependency.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Markus Lehtonen [Thu, 21 Jan 2016 17:46:41 +0000 (19:46 +0200)]
build-perf-test.sh: add eSDK testing
Add simple initial eSDK test. Currently, only download size and
installation time of eSDK is measured. The eSDK to be tested is
generated from the same image that the other tests are run for. This
patch will add two new fields to the global results log and that needs
to be taken into account when examining the results.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Fri, 22 Jan 2016 03:23:37 +0000 (19:23 -0800)]
gstreamer1.0-plugins.inc: drop obsolete ${S}/po/Makefile.in.in workaround
The workaround was originally added to the 0.10.x gst-plugins recipes
in late 2012 [1] and the root cause seems to have been fixed upstream
in late 2013 [2] (ie prior to the 1.4.0 release).
Andre McCurdy [Sat, 23 Jan 2016 01:26:03 +0000 (17:26 -0800)]
sqlite3.inc: dynamically link the sqlite3 command-line utility
By default, the sqlite3 command-line utility will be statically linked
with sqlite3. For OE, dynamic linking is probably more appropriate and
can be enabled by configuring with "--disable-static-shell".
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Sat, 23 Jan 2016 01:26:00 +0000 (17:26 -0800)]
sqlite3.inc: fix readline PACKAGECONFIG
Make it clearer that readline support for the target build is now
disabled by default.
Since readline dependencies and configure options are now handled by
PACKAGECONFIG, avoid duplicating readline in DEPENDS or duplicating
--disable-readline in EXTRA_OECONF.
Also add --disable-editline to EXTRA_OECONF to avoid a potential
floating dependency, since the configure script checks for libedit
before it checks for libreadline.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jan 2016 17:17:10 +0000 (17:17 +0000)]
uninative: Fix conflicts with normal sysroot
Currently this code installs into the standard sysroot, however this causes
some conflicts when linking since the linker can look specifically for
versioned .so files (e.g. like libpthreads.so.0). This breaks builds
of util-linux-native for example.
The easiest solution is to install uninative into its own separate sysroot.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jan 2016 13:05:21 +0000 (13:05 +0000)]
uninative: Add fetch capability
Originally, the idea was that the init environment would handle
fetching or providing the binary shim that uninative needs.
This turns out to be ugly, especially when you consider proxy
environments and so on getting involved. Instead, lets therefore
support our fetcher which already handles all this.
The distro is expected to setup configuration like:
Richard Purdie [Fri, 22 Jan 2016 12:58:11 +0000 (12:58 +0000)]
pixz: Add 1.0.6
xz gives better compression results than bzip/gz but is often slower.
Using parallel compression mitigates this somewhat and is particularly
useful for the SDK.
Whilst xz does have some parallel support, pixz appears to perform better
and supports parallel decompression as well as a simpler command line.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 22 Jan 2016 12:59:35 +0000 (12:59 +0000)]
xz: Allow to work with ASSUME_PROVIDED xz-native
At some point we may want to add xz-native to ASSUME_PROVIDED. This
allows that to work whilst still allowing access to liblzma for
those things which need it (e.g. pixz).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 12:00:01 +0000 (01:00 +1300)]
recipetool: create: better fix for fetch error handling
I was a little bit hasty in OE-Core revision c2cc5abe34169eae92067d97ce1e747e7c1413f5 - it turns out BitBake's
fetcher code is not consistent in whether it logs something useful or
not; when fetching from an http URL it does but with a git repository
it doesn't. In advance of any major reworking of fetch error handling in
BitBake, let's just print the text of the exception and then we know we
have shown something to the user.
Additionally, we were only catching FetchException here but there are
several other classes of exception that the fetcher can raise (e.g.
MalformedUrl); catch the parent BBFetchException class instead so we
avoid tracebacks for those other classes as well.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:59 +0000 (00:59 +1300)]
recipetool: create: extract SRC_URI from local git repositories
If you specify a local directory which happens to be a git repository
with an origin remote (and it is in fact remote), we can use that for
SRC_URI rather than leaving it blank in the recipe.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:58 +0000 (00:59 +1300)]
devtool / recipetool: support specifying a subdirectory within the fetched source
Sometimes you don't want to build an entire project, just a subdirectory
of it; add a --src-subdir option to make that easier. (We still look for
a single subdirectory in what gets unpacked, e.g. what you might find
within a tarball, so whatever you specify with this option is added onto
the end of that.)
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:56 +0000 (00:59 +1300)]
gen-lockedsig-cache: copy correct native sstate into ext SDK
When constructing the sstate-cache directory for the extensible SDK,
we were copying in any matching native sstate packages, and as the
signature doesn't actually change when the distro changes (since
NATIVELSBSTRING is just a path separator for the artifacts and is not
part of the signature) we ended up copying duplicated packages when the
distro changed e.g. upon host distro upgrade. Only search in the
NATIVELSBSTRING-named subdirectory for native packages and the issue
goes away.
Fixes [YOCTO #8885].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:55 +0000 (00:59 +1300)]
toolchain-shar-extract.sh: improve behaviour when xz is not installed
Currently if xz isn't installed when installing the SDK you get an error
from tar and then the SDK installation continues, cascading into a load
of other failures since files are missing. Add a check to see if xz can
be run and error out if it can't, and also exit if tar fails.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:54 +0000 (00:59 +1300)]
classes/populate_sdk*: add dependencies on script files
If the script files we use to construct the SDK installer change then
that really ought to trigger re-execution of the do_populate_sdk(_ext)
task, so add file-checksums varflags to ensure that happens.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:53 +0000 (00:59 +1300)]
classes/populate_sdk_ext: drop ext-sdk-prepare.py when installing
At the end of the extensible SDK installation, if we've successfully
prepared the build system then we don't need ext-sdk-prepare.py. I had
thought earlier that this would be used when updating, but a different
mechanism was needed there so this script isn't used for that.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:52 +0000 (00:59 +1300)]
devtool: add sdk-install subcommand
Add the ability to install additional pre-built items (from shared
state) into the extensible SDK. This can already be done implicitly by
adding something to DEPENDS within a recipe you're working on and then
running "devtool build", but it's useful to be able to explicitly
install things particularly if you're using the extensible SDK as a
traditional toolchain.
Note that for this command to be useful you need to have SSTATE_MIRRORS
set in your SDK configuration, and that mirror needs to be populated
with sstate artifacts for recipes you wish to be able to install.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:51 +0000 (00:59 +1300)]
devtool: sdk-update: improve SDK update process robustness
Make the following improvements to the SDK update process:
* Use a manifest file with sha256sums to track files other than sstate
and metadata that we need to update - e.g. conf files. This allows us
to handle where files such as auto.conf may or may not be present,
as well as the configuration changing without affecting task signatures
- we still want the config files copied in that case rather than it
saying nothing needs to be done.
* Write the SSTATE_MIRRORS_append to site.conf rather than local.conf
so that local.conf remains static (since we don't want to trigger an
update every time). Also, If there is an SSTATE_MIRRORS value already
set in the configuration we can skip this and assume it contains the
needed packages.
* Allow the update process to be run in any directory, don't assume
we're already at the base of the SDK
* Where practical, fetch remote files into a temporary location and
then move them to the desired location at the end, to avoid a
failed update leaving the SDK in a broken state.
* Update all installed do_populate_sysroot / do_packagedata tasks
instead of using the SDK targets. This ensures any item installed
through dependencies after installation (e.g. when running
"devtool build") won't go stale.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
* Use tempfile.mkdtemp() instead of hardcoding temp dir
* Set a variable early for the temp locked sigs file and use that
everywhere
* Delete the temp dir at the end
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:49 +0000 (00:59 +1300)]
devtool: build: ensure pkgdata is written out
When you run devtool build, you need to have the pkgdata written out at
the end, so that if what you're adding is a library and the next thing
you add is something that depends on that library, the necessary
information to map the dependency back to the recipe is present. In
practical terms all this means is we need do_packagedata to run in
addition to do_populate_sysroot.
This does mean that do_package needs to run which wasn't running before,
and that means that the few package QA tests that run within do_package
such as installed-vs-shipped will now be run. This may be a bit
bothersome, and prompted a fix for one of our oe-selftest tests as a
result, but I don't see an easy way around it. Ultimately if you care
about using the recipe in an image you'll need to fix any such errors
anyway.
Fixes [YOCTO #8887].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Fri, 22 Jan 2016 11:59:48 +0000 (00:59 +1300)]
classes/populate_sdk_ext: add option to bring in pkgdata for world
Add a variable SDK_INCLUDE_PKGDATA which you can set to "1" to include
pkgdata for all recipes in the world target. There are a couple of uses
for this:
1) If you use "devtool add" to add a recipe that builds something which
depends on anything in world, the dependency can then be correctly
mapped to the recipe providing it and that recipe can be added to
DEPENDS, since we have the pkg-config and shared library dependency
data within pkgdata.
2) You'll be able to search for these recipes and any files they
package for the target with "devtool search" since that also uses
pkgdata
This of course assumes you've tailored world through EXCLUDE_FROM_WORLD
to only include recipes you'd want built in your distro, but I think
that's a reasonable assumption; failing that there is a
WORLD_PKGDATA_EXCLUDE variable that you can set to exclude any recipes
you don't want.
Note that this patch relies on functionality implemented in a recent
BitBake patch and will not work without it.
Implements [YOCTO #8600].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Khem Raj [Fri, 15 Jan 2016 18:38:52 +0000 (18:38 +0000)]
syslinux: Set LD to avoid using build host ld
base ld is used during link phase and since its x86 so it
works mostly, however it can cause problems depending upon
build host. During cross build we should use cross linker
Fixes errors e.g.
Khem Raj [Wed, 13 Jan 2016 19:40:52 +0000 (19:40 +0000)]
gpgme: Define __error_t_defined on musl
Configure looks for error_t in errno.h and musl does not define error_t
if its not defined then configure emits a definition to be used, now we
also use standalone argp implementation from glibc on musl based systems
which defines this in argp.h as well so now we have it defined twice
and errors like
|
/a/builder/mnt/build-oe/tmp-musl/sysroots/raspberrypi2/usr/include/argp.h:40:13:
error: cannot combine with previous 'int' declaration specifier
| typedef int error_t;
| ^
| ../config.h:257:17: note: expanded from macro 'error_t'
| #define error_t int
| ^
| 1 error generated.
are seen. argp.h definition is conditional on __error_t_defined which is
true here so lets define it on musl
Khem Raj [Sun, 10 Jan 2016 21:00:02 +0000 (21:00 +0000)]
musl: Create ld.so as a relative symlink
We have rootfs intercepts which fail on musl because ld.so on musl is an
absolute symlink to /usr/lib/libc.so and then it fails to load when run
with qemu user mode.
Fixes rootfs failures like
WARNING: The postinstall intercept hook 'update_pixbuf_cache' failed
(exit code: 255)! See log for details!
WARNING: The postinstalls for the following packages will be postponed
for first boot: libgdk-pixbuf-2.0-loader-jpeg li
bgdk-pixbuf-2.0-loader-png libgdk-pixbuf-2.0-loader-gif
libgdk-pixbuf-2.0-loader-xpm
WARNING: [log_check] core-image-sato: found a warning message in the
logfile (keyword 'WARNING:'):
[log_check] WARNING: The postinstall intercept hook
'update_pixbuf_cache' failed (exit code: 255)! See log for details!
WARNING: [log_check] core-image-sato: found a warning message in the
logfile (keyword 'WARNING:'):
[log_check] WARNING: The postinstalls for the following packages will be
postponed for first boot: libgdk-pixbuf-2.0-lo
ader-jpeg libgdk-pixbuf-2.0-loader-png libgdk-pixbuf-2.0-loader-gif
libgdk-pixbuf-2.0-loader-xpm
Khem Raj [Sun, 10 Jan 2016 19:15:20 +0000 (11:15 -0800)]
fts: Fix linker hash-style option
pass --hash-style explicitly to linker
and ensure that mips does not use gnu hash style
This inoculates fts when compiler does not pass
hash style option e.g. clang
Khem Raj [Sun, 10 Jan 2016 02:40:09 +0000 (18:40 -0800)]
pcmciautils: Fix parallel build and include sys/types.h
parallel build patch was removing dependencies on .c files mistakenly
just adding src/yacc_config.h to dependencies in existing rule should
have fixed the original build race
include sys/types.h in lex_config.l for getting u_long definition
Khem Raj [Wed, 6 Jan 2016 22:18:00 +0000 (14:18 -0800)]
iputils: Use member based initialization for mrghdr struct
Fix build with musl
uclibc and glibc dont agree on structure of the struct, musl rightly
adds padding elements, so when doing anonymous initialization struct
elements gets wrongly mapped on 64bit arches
backtrace is glibc only feature on linux so check for glibc before using
it
Clangs clear_cache has different signature than gcc provided builtin
make it compatible for both
