When using sstate, or performing an incremental build any change to
the do_stash_locale() will cause a build failure because
do_stash_locale() was destroying the results obtained from the
do_install() with several mv operations. A recent change to
do_stash_locale() for a different problem illustrated a number of
build failures for users in the community.
To fix the problem, do_stash_locale() must use copy operations instead
of the mv operations. Because this is changed to a copy, the sysroot
and package stage need to remove the files that would have been
previously removed. The correct "fixup" code to deal with the removal
already existed in the previous do_poststash_install_cleanup(). All
that needed change was the path to where to remove the files
from the sysroot and package stages.
In order to force a re-compilation of glibc some unused white space
was removed from do_compile() for glibc. I could not find any other
way around this and we don't want to have all the community folks to
have another iteration where they have to remove their tmp directories
or purge some portion of the sstate. It also makes this change
bisectable. If the change to the glibc is not included, it will fail
with the following message:
=====
| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip: /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: No such file or directory
=====
After this one time change I tested changing only the
do_stash_locale() function and it now works well because it is
deterministically operating off the sstate data or a local build.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mike Crowe [Tue, 9 Jul 2019 08:25:09 +0000 (09:25 +0100)]
image.bbclass: Only append to IMAGE_LINK_NAME if it was already set
create_symlinks does not create any links if IMAGE_LINK_NAME is empty.
Unfortunately, setup_debugfs_variables unconditionally appends '-dbg' which
results in a previously-empty IMAGE_LINK_NAME containing just '-dbg'. Let's
check that it's not empty before appending.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 8 Jul 2019 23:46:50 +0000 (00:46 +0100)]
buildhistory: filter out the unexpected prefix for native/cross sysroots
For various technical reasons, native and cross builds have a prefix that
includes the full path to the sysroot. As these are stripped away before the
files are used in the sysroot, we should also filter them out of the
buildhistory report. This both removes noise when sharing a buildhistory
repository between different build directories, and improves the accuracy of the
reports.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 8 Jul 2019 23:46:43 +0000 (00:46 +0100)]
buildhistory: don't output ownership for the sysroot
As the sysroot isn't ran inside pseudo the ownership is whoever is running the
builds. In a setup where multiple builders all contribute to a shared
buildhistory writing the ownership data isn't useful, so just replace it with "-
-".
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
efibootmgr: Pass correct flags to compiler from pkg-config
efivar.h is in usr/include/efirvar directory so it should be
added to include search path via -I to compiler cmdline to fix
make[1]: *** No rule to make target 'efivar.h', needed by 'efibootmgr.o'. Stop.
| make[1]: *** Waiting for unfinished jobs....
When running clang to generate dependencies -MM -MG -MF it still
parses the compile unit and complains if certain header is not found
where as gcc does not do that, hence the compile error is only seen
when compiling with clang.
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Tue, 9 Jul 2019 10:50:58 +0000 (11:50 +0100)]
libsndfile1: remove redundant autoconf seeding
Twelve years ago libsndfile was badly detecting large file handling and
generating bad code[1]. The detection code in libsndfile has had many fixes
since then and this isn't needed anymore (verified by comparing config.h when
built for qemuarm).
Ross Burton [Tue, 9 Jul 2019 10:50:57 +0000 (11:50 +0100)]
libsndfile1: disable use of sqlite3 by default
sqlite3 is only used by the regression testing tool, which is of limited use
unless you're the developer of libsndfile. Add a PACKAGECONFIG for this, but
disable by default.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 6 Jul 2019 16:46:28 +0000 (17:46 +0100)]
core-image-sato-sdk-ptest: Reduce image padding size due to bootimg 4GB limit
This image continues to run out of space on the autobuilder, tweak it a bit
further now the image space requirements were reduced after various ptest
fixes to avoid the error.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
go.bbclass: separate the ptest logic to go-ptest class
The current go class includes ptest logic by default
and will make the recipe which inherits go class
to support ptest automatically though maybe the
recipe which inherits go class doesn't plan to
support the ptest.
So separate the ptest logic to another specified
class go-ptest to make the recipe which needs to
inherit go class more flexible with regards to
ptest support.
Signed-off-by: Mingli Yu <mingli.yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
If the NVD url is not accessible, print a warning on top of the CVE
report, and continue. The database will not be fully updated, but
cve_check can still run on the previous database.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
devtool.py: track to clean devtool.conf in test_create_workspace
`devtool create-workspace' would create devtool.conf, so track to
clean it up. Otherwise, this devtool.conf file would mess things up.
e.g.
oe-selftest -r devtool && oe-selftest -r devtool -j 2
AssertionError: '/PROJ_DIR/build-selftest-st-15753/workspace/conf/layer.conf' does not exist : Workspace directory not created
This is because the devtool.conf is also copied to build-selftest-st-xxxx
directory, resulting in devtool to create and use workspace specified
in this file.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:09 +0000 (13:16 -0700)]
glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
The do_stash_locale was not working consistently across the 4 build
configurations and the multilib, usrmerge configuration would fail
entirely with the obscure message:
| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip: /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: No such file or directory
| WARNING: /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.124690:1 exit 1 from 'gunzip $i'
Here is the 4 build configurations without the patch applied:
A) x86-64 no multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
909
B) x86-64 no multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
909
C) x86-64 multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
885
D) x86-64 multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
864
The issue here is that all the moves should be processed first, then a
copy should be made of the lib directories, but only in the case they
are different when using the usrmerge feature. Even though the build
worked for the multilib configuration without usrmerge, the content
was not the same.
After applying the patch the same number of files are in all the
configurations. The list of files was also diffed, after normalizing
the directory names to ensure all the correct files were copied.
Ultimately there are probably additional files that should be pruned
from what is copied to the stated_locale, but the purpose of this
patch is make it 100% consistent between the build types and fix the
builds.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:08 +0000 (13:16 -0700)]
glibc-locale: Fix build error with PACKAGE_NO_GCONV = "1"
When the PACKAGE_NO_GCONV is set to 1 an empty directory is left behind from the do_install rule:
=====
ERROR: glibc-locale-2.29-r0 do_package: QA Issue: glibc-locale: Files/directories were installed but not shipped in any package:
/usr/lib
/usr/lib/locale
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
glibc-locale: 2 installed and not shipped files. [installed-vs-shipped]
ERROR: glibc-locale-2.29-r0 do_package: Fatal QA errors found, failing task.
=====
The simple fix is to prune the empty directory.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:07 +0000 (13:16 -0700)]
psmisc: Fix dependency for USE_NLS=no
When using USE_NLS="no" in the local.conf psmisc will fail to
compile as follows:
| autoreconf: Entering directory `.'
| autoreconf: running: autopoint --force
| autoreconf: failed to run autopoint: No such file or directory
| autoreconf: autopoint is needed because this package uses Gettext
| ERROR: autoreconf execution failed.
This is because the gettext.bbclass returns gettext-minimal-native for
the host dependency which does not include autopoint. The autopoint
utility is required to build psmisc, so it needs to list
gettext-native as a dependency.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests the bitbake --skip-setscene option to verify that it correctly
reuses tasks previously restored from sstate, but doesn't execute new
sstate tasks
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Tue, 2 Jul 2019 08:32:40 +0000 (04:32 -0400)]
rng-tools: fix rngd blocks system shutdown
fix-rngd-fail-to-stop.patch is added to fix rngd blocks system shutdown
issue. But it causes rngd doesn't release CPU and causes 100% CPU
usage, so drop it.
The block shutdown issue is caused by comit [7cb64b9fe1 rng-tools: start
earlier in the boot process] which updated rngd.service. Revert the
modification of rngd.service.
Update sed expressions in do_install as well which fails to replace
second match in one line.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:24:00 +0000 (16:24 +1200)]
devtool: upgrade: fix handling of errors parsing upgraded recipe
As part of upgrading a recipe we create the upgraded recipe file in the
workspace and then try to parse it so we can then make further
modifications. If for some reason that parsing fails then the failure
was not being handled very well - the broken recipe was being left in
place, breaking parsing until it was removed by hand. Fix that by adding
a call to the cleanup function, and fix the following issues:
* Fix the cleanup function which doesn't look like it has ever worked
due to a typo in the function call
* Fix double-printing the error message
* Remove usage of DevtoolError in this case (DevtoolError is for simple
usage errors, not this kind of issue which may be the result of a
bug).
We're still printing a traceback in this scenario but at least it
doesn't break the build system requiring manual cleanup. I also
introduced a command-line option to preserve the broken upgraded recipe
file(s) for debugging purposes.
(The reproducer for this is "devtool upgrade libnewt-python", however
you need to check out revision b82ea144e144671d3f64c0785ba4beafe905cd4f
or earlier since that recipe has now been absorbed into the libnewt
recipe. The libnewt-python recipe was causing an issue with the upgrade
because it actually included the libnewt recipe using ${PV} in the
include statement, and of course PV was changing in the upgrade.)
Fixes [YOCTO #13404].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:23:59 +0000 (16:23 +1200)]
recipetool: ignore zero-length setup.py files
If a setup.py file exists it ought to have something in it before we
consider the source tree to be a Python module and treating it as such.
(A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz -
it's not clear why this has a zero-length setup.py in it but we should
pay no attention to it.)
Fixes [YOCTO #12923].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:12:47 +0000 (16:12 +1200)]
list-packageconfig-flags: print PN instead of P
P (which is ${PN}-${PV}) isn't terribly useful in this context - we
don't really care what the version is, but we do want to know what the
recipe is so we can find it or set PACKAGECONFIG_pn-<PN> in our
configuration, so display ${PN} instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:12:45 +0000 (16:12 +1200)]
scripts/contrib/ddimage: replace blacklist with mount check
The blacklist, whilst previously useful for safety, is now becoming
obsolete - on my current system, the main storage is at /dev/nvme* and
if I plug in a USB stick it shows up as /dev/sdb which was previously
blacklisted. To make this more flexible, remove the blacklist and
instead check if the specified device is mounted, has a partition
that is mounted, or is otherwise in use according to the kernel, and
show an appropriate error and quit if so.
To make this robust, also ensure we handle where the specified device is
a symlink to another device.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 02:35:55 +0000 (14:35 +1200)]
libcap-ng: do not use symlink to share files with libcap-ng-python
I'm not sure what's going on but having this symlink present is causing
git problems rebasing just by being present; deleting it and checking it
out again does not fix it. In any event this is not the standard way of
sharing files between recipes in the same directory - extending
FILESEXTRAPATHS is, so use that method instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changqing Li [Mon, 1 Jul 2019 05:04:53 +0000 (13:04 +0800)]
mdam: fix mdmonitor start up failure
1. recently, mdadm has changed to use service file under srcdir,
so remove the one not be used.
2. add -y option to fix below problem
mdadm: No mail address or alert command - not monitoring
Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check the existence of systemd before using systemctl to preset units.
This is because even if 'systemd' is in DISTRO_FEATURES, it's possible
that systemd is not even installed. e.g. container-test-image in
meta-selftest layer.
As systemd DEPENDS on systemd-systemctl-native, the existence of systemd
also ensures the existence of systemd-systemctl-native.
This would fix the following test case when using systemd as the init
manager.
Li Zhou [Mon, 1 Jul 2019 02:07:16 +0000 (10:07 +0800)]
bc: dc: fix exit code of q command
The exit code for "echo q | dc" is 1 for dc-1.4.1;
while the exit code for "echo q | dc" is 0 for dc-1.4.
Here is the answer from ken@gnu.org:
dc-1.4 was right. There was a rewrite of a chunk of code for 1.4.1 to
fix a corner case in the Q command, and somehow the placement of the
clean-up label for the 'q' command got misplaced on the error-handling
branch instead of the clean-exit branch. The patch below fixes this
(it is committed for whenever the next bc/dc release gets made).
Thanks for the report,
--Ken Pizzini
Signed-off-by: Li Zhou <li.zhou@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 28 Jun 2019 15:45:53 +0000 (16:45 +0100)]
package: Build pkgdata specific to the current recipe
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
style.css was set to write-only during build which means it wasn't actually
installed or packaged (with only webkitgtk producing a build failure!),
which in turn means generated documentation was broken.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
