Richard Purdie [Fri, 10 Nov 2017 14:41:32 +0000 (14:41 +0000)]
testimage: Ensure full logs are shown for failures
Currently, the fact an error message is shown means the rest of the
task logs are suppressed. In this case we don't want that as it hides
the real errors and useful information. Therefore override this behaviour.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:17:17 +0000 (18:17 +0000)]
oeqa: Clean up logger handling
The logger handling in oeqa was confused at best. This patch:
a) Passes in a logger through various qemu runner pieces
b) Uses that logger consistently in the code
c) Creates a logger for QemuRunner outside the bitbake namespace
meaning we don't conflict with the tinfoil logging changes
The result of this is more consistency. For runtime tests in testimage,
the logs always contain the debug info, nothing is shwon on the console.
For the oe-selftests, logs are intercepted and only shown if the test
fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:15:02 +0000 (18:15 +0000)]
testimage: Pass the logger into OERuntimeTestContextExecutor.getTarget()
I have no idea why we didn't do this but it means the code has nowhere
to log to unless we do this. This means we can then use the logger
to log data to the task logs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:13:24 +0000 (18:13 +0000)]
oeqa/qemurunner: Use logger.debug, not logger.info
Bitbake logs info messages to the console. These messages are really
there as debugging information. At the debug level, they will be shown
in failure logs and in the task logs but not on the console which
is what we want in this case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 14:03:28 +0000 (14:03 +0000)]
oeqa/runqemu: Only show stdout/stderr upon test failure
In general we don't need to see the output of runqemu however if it fails
we do. Use the buffer option that already exists in TestResult but allow
us to trigger it on a per test basis.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Nathan Rossi [Thu, 9 Nov 2017 11:09:25 +0000 (21:09 +1000)]
busybox.inc: Fix bashism in compile, which fixes sh being suid
Fix the bashism in the suid check. This ensures that the check works
correctly on hosts that default sh to e.g. dash. If this check fails the
suid shell workaround does not remove sh from the suid binary and
results in the target system containing a busybox.suid with sh as well
as /bin/sh -> /bin/busybox.suid.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Cc: Andrej Valek <andrej.valek@siemens.com> Cc: Radovan Scasny <radovan.scasny@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Fri, 27 Oct 2017 09:43:51 +0000 (17:43 +0800)]
gcc: fix miscompilation on mips64
We've observed strange behaviour of `systemctl status <xxx> on qemumips64.
The output of the command is like `systemctl show <xxx>', which is incorrect.
This patch is from gcc bugzilla's attachment.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81803
The patch hasn't been merged into gcc. But it does solve the above problem.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Markus Lehtonen [Wed, 25 Oct 2017 11:39:38 +0000 (14:39 +0300)]
oe-build-perf-report-email.py: add images as MIME objects
Add images as separate MIME objects instead of directly embedding images
in the html (as base64 encoded pngs). This makes the emails better
suited for certain email servers/clients.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Tom Rini [Thu, 21 Sep 2017 17:46:17 +0000 (13:46 -0400)]
wic: Update canned-wks for systemd to use UUID everywhere
With systemd, the mounting of the swap partition is handled via systemd
and will mount it, regardless of if PARTUUID is parsed or not. systemd
has a runtime dependency on util-linux-mount so PARTUUID for regular
mount points will be handled correctly. Make all partitions that we add
to the image make use of UUIDs for maximum portability.
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Tom Rini [Thu, 21 Sep 2017 17:46:16 +0000 (13:46 -0400)]
wic: When using --use-uuid make sure that we update the fstab with PARTUUID
When we have been told to use the UUID we should also update the fstab
to make use of PARTUUID instead of hard-coding the device in question.
This will make the resulting image much more portable.
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Chen Qi [Fri, 27 Oct 2017 09:43:50 +0000 (17:43 +0800)]
systemd: remove useless options for mips4
Looking back the history, we had problem with systemd on qemumips64
which is also related to compilation flags. We solved that by using
tweaking FULL_OPTIMIZATION for mips64 to have "-fno-tree-switch-conversion
-fno-tree-tail-merge".
Now systemd has been upgraded to 234, and we don't have the above problem
any more, thus removing these flags.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ming Liu [Mon, 2 Oct 2017 05:25:42 +0000 (07:25 +0200)]
libsolv: fix a kernel-devsrc installation issue
We encountered a problem when installing kernel-devsrc package on a
intel-x86 target, as follows:
$ dnf install kernel-devsrc
| Installing : kernel-devsrc-1.0-r0.0.intel_corei7_64 1/1
| failed loading RPMDB
| The downloaded packages were saved in cache until the next successful transaction.
| You can remove cached packages by executing 'dnf clean packages'.
It can be fixed by increasing MAX_HDR_CNT and MAX_HDR_DSIZE in libsolv
per test.
Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Wed, 1 Nov 2017 00:28:34 +0000 (00:28 +0000)]
python: add PACKAGECONFIG for Berkeley DB module
The bsddb module is deprecated and requires an old version of Berkeley DB that
some may be unhappy with even shipping, so expose a way to disable the module.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Wed, 1 Nov 2017 13:52:19 +0000 (13:52 +0000)]
db: change types to avoid headers changing between architectures
Triggered by looking at why Python doesn't find db.h (because it greps db.h for
a regex, and db.h is actually a oe_multilib_header wrapper) I realised that the
only reason we have to oe_multilib_header db.h is because one typedef is
different between 32-bit and 64-bit architectures.
However, the typedef is for a 64-bit integer so instead of using long (64-bit)
or long long (32-bit), just use int64_t. Some of the overly complicated
configure tests need to be deleted after this change but that is safe as we're
building in a controlled environment and can assume int64_t exists.
With this done the header doesn't change between architectures, and it doesn't
need to be wrapped by oe_multilib_header.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Erik Botö [Mon, 6 Nov 2017 18:13:06 +0000 (10:13 -0800)]
sshcontrol.py: in copy_to() always use scp
The current implementation is broken when the localpath is a link.
Then only a symlink would be created on the target, instead of copying
the actual file.
[YOCTO #11524]
Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Erik Botö [Mon, 6 Nov 2017 18:13:04 +0000 (10:13 -0800)]
masterimage.py: fix stop()
The stop() function is called in the context of the masterimage,
so self.master should be used instead of self.connection which is
undefined at that time.
[YOCTO #11524]
Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Erik Botö [Mon, 6 Nov 2017 18:13:03 +0000 (10:13 -0800)]
masterimage.py: fix issue with calling reboot on masterimage/DUT
On systemd systems calling reboot over an ssh connection doesn't
return as expected causing an exception, therefore wrap the call
to reboot in order to avoid this issue.
Also sync the filesystems before rebooting cause otherwise, it will be
done as part of the reboot and could take a very long time and testimage
will fail to access the machine. This issue was observed consistently with
one of our rootfs at Pelagicore.
[YOCTO #11524]
Signed-off-by: Erik Botö <erik.boto@pelagicore.com> Signed-off-by: Stephano Cetola <stephano.cetola@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Joshua Lock [Wed, 4 Oct 2017 16:22:48 +0000 (17:22 +0100)]
lib/oeqa/selftest/cases/gotoolchain: add selftest for the Go toolchain
Add a simple test case to being testing of the Go toolchain:
1) build meta-go-toolchain
2) create a temp directory and install the generated Go toolchain within
3) fetch an archive of the Go Dep tool
4) create an appropriately laid out GOROOT and inflate the dep archive there
5) build the dep command with the SDK's Go toolchain and check it returned
successfully.
[YOCTO #12152]
Signed-off-by: Joshua Lock <joshua.g.lock@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa/core/loader: Make _built_modules_dict() support packages correctly
For test modules in a package, e.g., oelib.license, running
`oe-selftest -r oelib.license` or `oe-selftest -r
oelib.license.TestSimpleCombinations` would fail with a message that
the specified test cases could not be found. This was due to the
parsing in _built_modules_dict(), which failed to distinguish between
<package>.<module>.<class> and <module>.<class>.<testcase> and treated
both cases as the latter.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ming Liu [Tue, 26 Sep 2017 12:31:16 +0000 (14:31 +0200)]
image.bbclass: let do_image depend on do_populate_lic of EXTRA_IMAGEDEPENDS
The licenses of EXTRA_IMAGEDEPENDS recipes are being referenced in
image postcommand write_deploy_manifest, but a dependency is missing
between do_image and do_populate_lic of EXTRA_IMAGEDEPENDS recipes,
this leads some license files not present when write_deploy_manifest
runs, hence will cause build errors.
Fixed by letting do_image depend on do_populate_lic of
EXTRA_IMAGEDEPENDS recipes.
Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
externalsrc: fix ExpansionError if the source dir does not exist yet
The externalsrc class code assumes that the source directory
(EXTERNALSRC) exists before bitbake is called. Otherwise do_configure
will fail obviously since externalsrc does not fetch anything.
Commit 3ca6085729d9 ("externalsrc: Handle .git not being a directory")
changed this behaviour. Now on a missing EXTERNALSRC directory we get
a bb.data_smart.ExpansionError during _parsing_, way before
do_configure can be run.
This new behaviour creates two problems:
* First, there error message is very cryptic (and it's hard to
provide a better message since no task is ever run):
ERROR: ExpansionError during parsing /<...>/<...>.bb
Traceback (most recent call last):
bb.data_smart.ExpansionError: Failure expanding variable do_compile[file-checksums], expression was ${@srctree_hash_files(d)} which triggered exception FileNotFoundError: [Errno 2] No such file or directory: '<...>'
* Second, this prevents creating a class based on externalsrc that
automatically fetches the code in EXTERNALSRC before do_compile
runs.
Fix both problems by simply calling git with '-C ${EXTERNALSRC}'
instead of calling git inside the non-existing directory. This changes
from a bb.data_smart.ExpansionError to a
subprocess.CalledProcessError, which is in line with what's actually
going on: git is telling us it can't find the git dir.
Also remove a comment that does not apply anymore.
Patrick Ohly [Thu, 19 Oct 2017 07:13:26 +0000 (09:13 +0200)]
useradd-staticids: explain how to fix the the problem
When a distro uses useradd-staticids.bbclass and some developer
unfamiliar with the static ID mechanism tries to add a recipe which
needs new IDs, the resulting error or warning is typically not
something that the developer will understand.
Even experienced developers do not get enough information. They first
must find out whether the missing ID is for a system user or group,
then locate the file(s) in which the ID could be added. Both of this
is now part of the message:
ERROR: .../meta/recipes-extended/cronie/cronie_1.5.1.bb: cronie -
cronie: system groupname crontab does not have a static ID defined.
Add crontab to one of these files: /.../conf/distro/include/my-distro-group
The case that no file was found is also handled:
ERROR: .../meta/recipes-extended/cronie/cronie_1.5.1.bb: cronie -
cronie: system groupname crontab does not have a static ID defined.
USERADD_GID_TABLES file(s) not found in BBPATH: files/group
It would be nice if the error message could also list the range in
which a new ID needs to be allocated, but /etc/login.defs isn't
available at the time of creating the message, so that part is still
something that a developer needs to know.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Patrick Ohly [Tue, 26 Sep 2017 07:29:47 +0000 (09:29 +0200)]
useradd-staticids: skip recipes without static IDs
When enabling useradd-staticids.bbclass, one has to define static IDs
for all recipes in a world build, otherwise those without static IDs
generate parse errors or warnings, depending on USERADD_ERROR_DYNAMIC.
Defining unused IDs is a lot of work and clutters the passwd/group
file of a distro.
Distros which want to avoid this can now set USERADD_ERROR_DYNAMIC =
"skip" and recipes which would have triggered a message then silently
get disabled. Only trying to build them shows the error message:
$ bitbake apt
...
ERROR: Nothing PROVIDES 'apt'
ERROR: apt was skipped: apt - apt: username _apt does not have a static ID defined.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ming Liu [Fri, 13 Oct 2017 11:26:16 +0000 (13:26 +0200)]
lib/oe/terminal.py: use an absolute path to execute oe-gnome-terminal-phonehome
A flaw was found on my Ubuntu 14.04.5 LTS, on which that gnome-terminal is
the default terminal, when I run any of the tasks:
bitbake busybox -c menuconfig/devshell/devpyshell
bitbake virtual/kernel -c menuconfig/devshell/devpyshell
I got a error as follows:
"Failed to execute child process "oe-gnome-terminal-phonehome" (No such file or directory)"
Seems the environment of the process calling Popen is not passed to the
child process, this behaviour is a known issue in Python bug tracker:
http://bugs.python.org/issue8557
It could be fixed by using an absolute path instead per test.
Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
