devtool.py: track to clean devtool.conf in test_create_workspace
`devtool create-workspace' would create devtool.conf, so track to
clean it up. Otherwise, this devtool.conf file would mess things up.
e.g.
oe-selftest -r devtool && oe-selftest -r devtool -j 2
AssertionError: '/PROJ_DIR/build-selftest-st-15753/workspace/conf/layer.conf' does not exist : Workspace directory not created
This is because the devtool.conf is also copied to build-selftest-st-xxxx
directory, resulting in devtool to create and use workspace specified
in this file.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:09 +0000 (13:16 -0700)]
glibc/glibc-locale: Fix do_stash_locale to work with usrmerge and multilibs
The do_stash_locale was not working consistently across the 4 build
configurations and the multilib, usrmerge configuration would fail
entirely with the obscure message:
| DEBUG: Executing shell function do_prep_locale_tree
| tar: i18n: Cannot stat: No such file or directory
| tar: Exiting with failure status due to previous errors
| gzip: /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/locale-tree//usr/share/i18n/charmaps/*gz.gz: No such file or directory
| WARNING: /poky/build/tmp/work/core2-64-poky-linux/glibc-locale/2.29-r0/temp/run.do_prep_locale_tree.124690:1 exit 1 from 'gunzip $i'
Here is the 4 build configurations without the patch applied:
A) x86-64 no multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
909
B) x86-64 no multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
909
C) x86-64 multilibs, no usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
885
D) x86-64 multilibs, usrmerge
find ./tmp/work/*/glibc/2.29-r0/stashed-locale -type f |grep -v nscd.service |wc -l
864
The issue here is that all the moves should be processed first, then a
copy should be made of the lib directories, but only in the case they
are different when using the usrmerge feature. Even though the build
worked for the multilib configuration without usrmerge, the content
was not the same.
After applying the patch the same number of files are in all the
configurations. The list of files was also diffed, after normalizing
the directory names to ensure all the correct files were copied.
Ultimately there are probably additional files that should be pruned
from what is copied to the stated_locale, but the purpose of this
patch is make it 100% consistent between the build types and fix the
builds.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:08 +0000 (13:16 -0700)]
glibc-locale: Fix build error with PACKAGE_NO_GCONV = "1"
When the PACKAGE_NO_GCONV is set to 1 an empty directory is left behind from the do_install rule:
=====
ERROR: glibc-locale-2.29-r0 do_package: QA Issue: glibc-locale: Files/directories were installed but not shipped in any package:
/usr/lib
/usr/lib/locale
Please set FILES such that these items are packaged. Alternatively if they are unneeded, avoid installing them or delete them within do_install.
glibc-locale: 2 installed and not shipped files. [installed-vs-shipped]
ERROR: glibc-locale-2.29-r0 do_package: Fatal QA errors found, failing task.
=====
The simple fix is to prune the empty directory.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jason Wessel [Tue, 2 Jul 2019 20:16:07 +0000 (13:16 -0700)]
psmisc: Fix dependency for USE_NLS=no
When using USE_NLS="no" in the local.conf psmisc will fail to
compile as follows:
| autoreconf: Entering directory `.'
| autoreconf: running: autopoint --force
| autoreconf: failed to run autopoint: No such file or directory
| autoreconf: autopoint is needed because this package uses Gettext
| ERROR: autoreconf execution failed.
This is because the gettext.bbclass returns gettext-minimal-native for
the host dependency which does not include autopoint. The autopoint
utility is required to build psmisc, so it needs to list
gettext-native as a dependency.
Signed-off-by: Jason Wessel <jason.wessel@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Tests the bitbake --skip-setscene option to verify that it correctly
reuses tasks previously restored from sstate, but doesn't execute new
sstate tasks
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Tue, 2 Jul 2019 08:32:40 +0000 (04:32 -0400)]
rng-tools: fix rngd blocks system shutdown
fix-rngd-fail-to-stop.patch is added to fix rngd blocks system shutdown
issue. But it causes rngd doesn't release CPU and causes 100% CPU
usage, so drop it.
The block shutdown issue is caused by comit [7cb64b9fe1 rng-tools: start
earlier in the boot process] which updated rngd.service. Revert the
modification of rngd.service.
Update sed expressions in do_install as well which fails to replace
second match in one line.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:24:00 +0000 (16:24 +1200)]
devtool: upgrade: fix handling of errors parsing upgraded recipe
As part of upgrading a recipe we create the upgraded recipe file in the
workspace and then try to parse it so we can then make further
modifications. If for some reason that parsing fails then the failure
was not being handled very well - the broken recipe was being left in
place, breaking parsing until it was removed by hand. Fix that by adding
a call to the cleanup function, and fix the following issues:
* Fix the cleanup function which doesn't look like it has ever worked
due to a typo in the function call
* Fix double-printing the error message
* Remove usage of DevtoolError in this case (DevtoolError is for simple
usage errors, not this kind of issue which may be the result of a
bug).
We're still printing a traceback in this scenario but at least it
doesn't break the build system requiring manual cleanup. I also
introduced a command-line option to preserve the broken upgraded recipe
file(s) for debugging purposes.
(The reproducer for this is "devtool upgrade libnewt-python", however
you need to check out revision b82ea144e144671d3f64c0785ba4beafe905cd4f
or earlier since that recipe has now been absorbed into the libnewt
recipe. The libnewt-python recipe was causing an issue with the upgrade
because it actually included the libnewt recipe using ${PV} in the
include statement, and of course PV was changing in the upgrade.)
Fixes [YOCTO #13404].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:23:59 +0000 (16:23 +1200)]
recipetool: ignore zero-length setup.py files
If a setup.py file exists it ought to have something in it before we
consider the source tree to be a Python module and treating it as such.
(A counter-example is https://www.bro.org/downloads/binpac-0.50.tar.gz -
it's not clear why this has a zero-length setup.py in it but we should
pay no attention to it.)
Fixes [YOCTO #12923].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:12:47 +0000 (16:12 +1200)]
list-packageconfig-flags: print PN instead of P
P (which is ${PN}-${PV}) isn't terribly useful in this context - we
don't really care what the version is, but we do want to know what the
recipe is so we can find it or set PACKAGECONFIG_pn-<PN> in our
configuration, so display ${PN} instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 04:12:45 +0000 (16:12 +1200)]
scripts/contrib/ddimage: replace blacklist with mount check
The blacklist, whilst previously useful for safety, is now becoming
obsolete - on my current system, the main storage is at /dev/nvme* and
if I plug in a USB stick it shows up as /dev/sdb which was previously
blacklisted. To make this more flexible, remove the blacklist and
instead check if the specified device is mounted, has a partition
that is mounted, or is otherwise in use according to the kernel, and
show an appropriate error and quit if so.
To make this robust, also ensure we handle where the specified device is
a symlink to another device.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 2 Jul 2019 02:35:55 +0000 (14:35 +1200)]
libcap-ng: do not use symlink to share files with libcap-ng-python
I'm not sure what's going on but having this symlink present is causing
git problems rebasing just by being present; deleting it and checking it
out again does not fix it. In any event this is not the standard way of
sharing files between recipes in the same directory - extending
FILESEXTRAPATHS is, so use that method instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changqing Li [Mon, 1 Jul 2019 05:04:53 +0000 (13:04 +0800)]
mdam: fix mdmonitor start up failure
1. recently, mdadm has changed to use service file under srcdir,
so remove the one not be used.
2. add -y option to fix below problem
mdadm: No mail address or alert command - not monitoring
Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Check the existence of systemd before using systemctl to preset units.
This is because even if 'systemd' is in DISTRO_FEATURES, it's possible
that systemd is not even installed. e.g. container-test-image in
meta-selftest layer.
As systemd DEPENDS on systemd-systemctl-native, the existence of systemd
also ensures the existence of systemd-systemctl-native.
This would fix the following test case when using systemd as the init
manager.
Li Zhou [Mon, 1 Jul 2019 02:07:16 +0000 (10:07 +0800)]
bc: dc: fix exit code of q command
The exit code for "echo q | dc" is 1 for dc-1.4.1;
while the exit code for "echo q | dc" is 0 for dc-1.4.
Here is the answer from ken@gnu.org:
dc-1.4 was right. There was a rewrite of a chunk of code for 1.4.1 to
fix a corner case in the Q command, and somehow the placement of the
clean-up label for the 'q' command got misplaced on the error-handling
branch instead of the clean-exit branch. The patch below fixes this
(it is committed for whenever the next bc/dc release gets made).
Thanks for the report,
--Ken Pizzini
Signed-off-by: Li Zhou <li.zhou@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 28 Jun 2019 15:45:53 +0000 (16:45 +0100)]
package: Build pkgdata specific to the current recipe
This switches the code to build pkgdata specific to the current recipe
which means that its filtered to the recipes dependencies and can perform
better as we can drop the lockfile.
It uses a similar method to the staging code to do this, using BB_TASKDEPDATA
to construct a list of packagedata task output which this recipe should "see".
The original pkgdata store is left unaltered so existing code works.
The lock file was there to prevent files disappearing as they were read or as
directories were listed. Since we have a copy of the data and only access output
from completed tasks (as per their manifests), we can remove the lock.
The lock was causing starvation issues on systems with parallelism.
There was also a potential determinism problem as the current code could "see"
data from recipes which it doesn't depend upon.
[YOCTO #13412]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
style.css was set to write-only during build which means it wasn't actually
installed or packaged (with only webkitgtk producing a build failure!),
which in turn means generated documentation was broken.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
cve-update-db: do_populate_cve_db depends on do_fetch
To be able to populate NVD database on a fetchall
(bitbake <image> --run-all=fetch), set the do_populate_cve_db task to be
executed before do_fetch.
Do not get CVE_CHECK_DB_DIR, CVE_CHECK_DB_FILE and CVE_CHECK_TMP_FILE
variable because do_populate_cve_db can be called in a context where
cve-check class is not loaded.
Signed-off-by: Pierre Le Magourou <pierre.lemagourou@softbankrobotics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Thu, 27 Jun 2019 03:14:55 +0000 (11:14 +0800)]
devtool: warn user about multiple layer having the same base name
Currently `devtool finish RECIPE meta' will silently succeed even
if there are multiple layers having the same base name of 'meta'.
e.g. meta layer from oe-core and meta layer from meta-secure-core.
We should at least give user a warning in such case. With the patch,
we will get warning like below.
WARNING: Multiple layers have the same base name 'meta', use the first one '<PROJ_DIR>/oe-core/meta'.
WARNING: Consider using path instead of base name to specify layer:
<PROJ_DIR>/oe-core/meta
<PROJ_DIR>/meta-secure-core/meta
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joël Esponde [Thu, 27 Jun 2019 09:12:04 +0000 (11:12 +0200)]
package.bbclass: fix directories setuid and setgid bits
populate_packages relies on ``mkdir`` to both create a directory and set
its permissions. However, ``mkdir`` honors the ``umask`` value.
Therefore, some bits may be lost in the operation. In our case, the
setgid bit on the directories were lost.
This commit fixes this by having a distinct call to create the directory
and to set the permissions.
Signed-off-by: Jean-Tiare Le Bigot <jean-tiare.le-bigot@easymile.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 26 Jun 2019 16:24:19 +0000 (17:24 +0100)]
oeqa/runtime/syslog: Add delay to test to avoid failures
On a loaded builder we've seen the log message not make it to the log file
before the ssh command completes. Add a short delay to try and ensure
this does happen. There is unforunately no way to flush syslog in all
cases we test.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 24 Jun 2019 15:41:45 +0000 (16:41 +0100)]
sysklogd: Fix init script races
In testing we're seeing sysklogd fail to restart klogd since the original
process hasn't stopped before the new one is started. This means a restart
can result in no process running which is clearly not desireable.
Add extra code to ensure this works correctly. Busybox start-stop-daemon
seems particularly open to this kind of issue, the dpkg version maybe
less so if timeout options are used (which we don't use).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jon Mason [Tue, 25 Jun 2019 16:06:58 +0000 (12:06 -0400)]
oe_syslog.py: Handle syslogd/klogd restart race
syslogd and klogd can occasionally take too long to restart, which
causes tests to fail by starting before the log daemons are ready. To
work around this problem, poll for up to 30 seconds on the processes to
verify the old ones are killed and the new ones are up and running.
Similarly, add checks for rsyslogd and systemd-journald to possibly
catch issues with those daemons.
[YOCTO #13379]
Signed-off-by: Jon Mason <jdmason@kudzu.us> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Tue, 25 Jun 2019 06:45:37 +0000 (14:45 +0800)]
target-sdk-provides-dummy: add libperl.so.5 64bit
With postgresql added to IMAGE_INSTALL, we will get the following
error when building for 64bit BSPs.
Problem: package postgresql-11.3-r0.corei7_64 requires libperl.so.5()(64bit), but none of the providers can be installed
A previous patch has added libperl.so.5 to DUMMY_PROVIDES, but this
is not enough. Because for 64bit BSP, it should also provide libperl.so.5()(64bit).
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
$ bitbake core-image-minimal
update-alternatives: libtool has multiple providers with the same priority,
please check
/path/to/rootfs/usr/lib/opkg/alternatives/libtool for details
Both libtool and lib32-libtool have the same priority (as they're the same
recipe), so update-alternatives won't deterministically pick a provider. This
means you could end up with an image using a 32-bit pkgconfig and 64-bit
libtool, for example.
Make extended recipes reduce priority by 1 (or 2, 3 ... when there are multiple
variants in MULTILIB_VARIANTS) to fix the problem.
[YOCTO #13418]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Wed, 26 Jun 2019 07:40:22 +0000 (15:40 +0800)]
busybox: make postinst run firstly before update-alternatives
The update-alternatives.bbclass' postinst script runs firstly before other
postinst, but busybox needs set basic tools such as sed command firstly,
otherwise, update-alternatives doesn't work, so run busybox' postinst firstly
to fix the problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Wed, 26 Jun 2019 07:40:21 +0000 (15:40 +0800)]
update-alternatives.bbclass: run update-alternatives firstly in postinst script
Recipes like postfix run command newaliases in postinst, but newaliases is
installed as newaliases.postfix, it needs run update-alternatives to update it
to newaliases, so there was an error when installed postinst on target.
Fixed:
$ opkg install postfix
Configuring postfix.
///var/lib/opkg/info/postfix.postinst: line 4: newaliases: command not found
Run update-alternatives firstly will fix the problem.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
KERNEL_VERSION gets expanded at runtime to contain the real kernel
version. There is code to ensure the signatures are determinisic but
the multilib expansion code breaks this.
Exclude the variable from the datastore used for expansion to avoid this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 25 Jun 2019 14:21:45 +0000 (15:21 +0100)]
multilib_global: Fix multilib rebuild issue
Building lttng-modules for a "lib32" multilib, then changing to a "lib64"
multilib with "lib32" removed doesn't rebuild lttng-modules.
This is due to the multilib pieces in RPROVIDES being added after RecipeParsed
which is after the signatures are generated.
Changing this to RecipeTaskPreProcess allows the multilib components to be
accounted for correctly in the task hashes.
This addresses failures on the autobuilder seen in lib64-core-image-sato-sdk
builds where lttng-modules was being reused from qemux86 world build's lib32
version.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Way back in
http://lists.openembedded.org/pipermail/openembedded-core/2014-April/210138.html
a few of us discussed not exporting TARGET_LDFLAGS. There seemed to be
support for this idea, and I modified our tree to not do so. I then seem to
have dropped the ball. :( We've been running like that for over five years,
and not observed any problems.
It seems sensible to stop exporting TARGET_CPPFLAGS, TARGET_CFLAGS and
TARGET_CXXFLAGS too.
I've successfully compile-tested core-image-minimal and core-image-sato for
x86_64 and qemuarm64 with these changes.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- Drop 0001-libopkg-add-add-ignore-recommends-option.patch
- Drop 0001-opkg-add-target-for-testsuite-installation.patch
- Drop 0001-regress-issue72.py-resolve-paths-before-comparision.patch
- Remove test binaries tests\libopkg_test, leftovers from make dist
process
Signed-off-by: Alejandro del Castillo <alejandro.delcastillo@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Mon, 24 Jun 2019 16:27:11 +0000 (17:27 +0100)]
gtk-icon-cache: rename intercept to update_gtk_icon_cache
The intercept is called update_icon_cache which is vague: rename to
update_gtk_icon_cache to make it clearer what it is for, and add a comment
explaining what class caused it to be used.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
