Richard Purdie [Wed, 25 Jul 2018 13:39:40 +0000 (13:39 +0000)]
selftest/bblayers: Place the test layer directory in builddir
Placing the layer in meta means the directory is in an unclean state
which may influence other tests. Use our build directory instead
since we 'own' that. This helps keep oe-selftest parallelisation
clean.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 23 Jul 2018 16:22:55 +0000 (16:22 +0000)]
license_image: Fix race
The current code pokes into do_deploy manifests from do_image_complete when the
do_image_complete task may or may not depend upon the do_deploy tasks in question.
Often it gets lucky, sometimes it results in build failures.
To fix this, split the functionality to its own task which can have the correct
task dependencies. This means the data in BB_TASKDEPDATA is definitive, the other
code can be dropped, as can the IMAGE_EXTRATYPES do_populate_lic dependencies
from image.bbclass.
This fixes bugs which show up as:
NOTE: recipe linux-yocto-4.14.48+gitAUTOINC+d64aec9793_97c8063d2d-r0: task do_deploy: Started
ERROR: core-image-minimal-1.0-r0 do_image_complete: Error executing a python function in exec_python_func() autogenerated:
The stack trace of python calls that resulted in this exception/failure was:
File: 'exec_python_func() autogenerated', lineno: 2, function: <module>
0001:
*** 0002:write_deploy_manifest(d)
0003:
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 33, function: write_deploy_manifest
0029: 'w+').write(output)
0030:}
0031:
0032:python write_deploy_manifest() {
*** 0033: license_deployed_manifest(d)
0034:}
0035:
0036:python license_create_manifest() {
0037: import oe.packagedata
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 191, function: license_deployed_manifest
0187: # It is necessary to mark this will be used for image manifest
0188: man_dic[dep]["IMAGE_MANIFEST"] = True
0189: man_dic[dep]["PN"] = dep
0190: man_dic[dep]["FILES"] = \
*** 0191: " ".join(get_deployed_files(dep_dic[dep]))
0192: with open(os.path.join(lic_dir, dep, "recipeinfo"), "r") as f:
0193: for line in f.readlines():
0194: key,val = line.split(": ", 1)
0195: man_dic[dep][key] = val[:-1]
File: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/classes/license.bbclass', lineno: 289, function: get_deployed_files
0285: """
0286:
0287: dep_files = []
0288: excluded_files = []
*** 0289: with open(man_file, "r") as manifest:
0290: all_files = manifest.read()
0291: for f in all_files.splitlines():
0292: if ((not (os.path.islink(f) or os.path.isdir(f))) and
0293: not os.path.basename(f) in excluded_files):
Exception: FileNotFoundError: [Errno 2] No such file or directory: '/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/build-st-730/tmp/sstate-control/manifest-qemux86_64-linux-yocto.deploy'
This breaks uninative tarball since the call of relocate_sdk.py from uninative.bbclass
wasn't updated to account for this change. It isn't clear what value that code could
pass in and this isn't simple to fix so revert until a better fix can be found
that doesn't break uninative.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Mon, 23 Jul 2018 12:40:14 +0000 (14:40 +0200)]
oe-selftest: fix for changes to buildhistory-diff
Now we're not reporting the related fields (as of openembedded-core 8658b3677b9f7cb70806061c41570c709086ef05) we shouldn't expect to see
PR reported here since it's not monitored by buildhistory-diff. However,
with a bit of messing about we can check for the exact output that we
should now see as a result of the test changing PR to go backwards.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 20 Jul 2018 15:44:05 +0000 (15:44 +0000)]
package: Clean up getstatusoutput usage
Replace usage of oe.utils.getstatusoutput() with direct subprocess calls. The
call is just a wraper to the subprocess call of the same name and requires
the caller to handle exceptions themselves. We usually do this badly, failing
to show the output or the command or the return code.
Its much safer to rely on a call like subprocess.check_output() instead.
This also makes it easier to spot and remove cases where shell=True isn't
needed in a later cleanup.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 20 Jul 2018 10:25:17 +0000 (10:25 +0000)]
sstate/bitbake.conf: Use pigz if available
Currently the compression of sstate objects is single threaded. In the case
of ltp, this takes around 33s. If we add pigz into the list of non-fatal
HOSTTOOLS and then use if it available when building the sstate object,
this time drops to around 6s.
Since pigz is now widely available this is an optimisation we should
utilise.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 20 Jul 2018 08:45:28 +0000 (08:45 +0000)]
package: Refactor to remove isElf/is_elf function duplication
There are probably further cleanups needed here but this at least removes
the major code duplication between these two similar funcitons, keeping the
kernel module ".ko" extension check for efficiency to avoid opening and
reading file contents in the general case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 20 Jul 2018 08:32:49 +0000 (08:32 +0000)]
package: Allow parallel processing of shlib analysis
This function is a bit more invasive to add parallelism to but
allows the shlibs analysis to happen in multiple threads. In order
to return values correctly/safely the data types needed tweaking
to avoid lists and use immutable objects.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 20 Jul 2018 08:31:59 +0000 (08:31 +0000)]
package: Allow parallel processing of debug splitting
Adjust the code so that the splitting of debug symbols from files happens
in parallel. To to this we need to move some path handling code into the
main function and pass more parameters in.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 19 Jul 2018 20:59:05 +0000 (20:59 +0000)]
package: Drop subshell usage for dwarfsrcfile generation.
The command for running dwarfsrcfiles is simple and does not need a subshell
for each execution. By expanding out this function to use check_output()
from subprocess and a list of arguments, the shell overhead can be dropped.
For recipes with lots of files this gives a significant saving.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 19 Jul 2018 20:32:44 +0000 (20:32 +0000)]
package_ipk|deb: Use oe.utils.multiprocess_launch
The current code had broken exception handling due to the use of a
"traceback" variable as well as an import. Use the new library code
for this instead which reduces code duplication and has fixed/improved
exception handling.
The chdir code can be dropped since any directory changes are in other
processes now so there is no need for it here and the code no longer
changes directory.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 18 Jul 2018 12:54:05 +0000 (13:54 +0100)]
package: Rework PACKAGELOCK based upon sstate for do_packagedata
I think this lock dates from before we had sstate for do_packagedata.
Since WORKDIR is recipe specific and we write into WORKDIR, we no longer need
any write locks in the do_packagedata code itself, its handled by the sstate
task lock for the final copy in at the end. The final write lock can be simply
removed.
The only time we need read locking is when actually reading data from the
shared directory. We can therefore reduce the window the lock is held
significantly as well, hence improving the speed of packagedata tasks running
in parallel.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 19 Jul 2018 20:31:35 +0000 (20:31 +0000)]
utils: Add multiprocess_launch API and testcase
The current methods of spawning processes for parallel execution have
issues around collection of results or exceptions.
Take the code from package_ipk/deb, make it generic, add a results
collection mechanism, fix the exception handling and for it into a
standard library function.
Also add a test case which tests both the success and failure modes
of operation to stop this functionality regressiing again.
In particular, compared to multiprocess_exec, this fork off the parent
approach means we can pass in the datastore and functions work in the
same scope as the parent. This removes some of the complexities
found trying to scale multiprocess_exec to wider use.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
New version adds support for command line options
(-e and -- command) which can be used for launching
programs or running commands through matchbox-terminal.
Signed-off-by: Devarsh Thakkar <devarsht@xilinx.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ruslan Bilovol [Wed, 22 Nov 2017 11:20:05 +0000 (13:20 +0200)]
relocate_sdk.py: remove hardcoded SDK path
This patch removes hardcodes added to relocate_sdk.py
during SDK build, making it flexible and reusable.
Now default SDK path is passed to the script as
parameter rather then harcoded inside it.
This allows to reuse this script for multiple
relocations, and adds possibility to relocate
SDK multiple times
Signed-off-by: Ruslan Bilovol <rbilovol@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Tue, 17 Jul 2018 12:34:17 +0000 (14:34 +0200)]
oe-run-native: ensure arguments get quoted
If you pass quoted arguments to oe-run-native, they need to be passed
as-is to the underlying tool. Putting quotes around $@ ensures each
argument is quoted individually.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Tue, 17 Jul 2018 16:38:56 +0000 (17:38 +0100)]
bc: upgrade to 1.07.1
Relicensed to just GPLv3+, update LICENSE and checksums.
The build dependency for flex should be flex-native.
libmath.h is missing from the tarball (it was present in 1.06) and the
generation rules are not cross-friendly, so delete the rules and copy in a
pre-generated libmath.h.
Remove fix-segment-fault.patch as the fixes are now upstream.
Add PACKAGECONFIG for readline and libedit, defaulting to readline.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
package.bbclass: exclude packages as shlib providers
Some packages containing shared libraries might be registered
as shlib providers when they shouldn't (for example, the lib is for
their private use and must not generate any dependency).
EXCLUDE_FROM_SHLIBS is targeted at that, but it could be set
for entire recipe only.
This patch allows to set list of packages that are not candidates
for shlib providers.
Variable EXCLUDE_PACKAGES_FROM_SHLIBS is used, e.g.:
EXCLUDE_PACKAGES_FROM_SHLIBS = "glibc-ptest glibc-mtest"
* Dropped Patches and Reasons
0001-Hide-__start_BUS_ERROR_MAP-and-__stop_BUS_ERROR_MAP.patch
This patch is dropped because the problem has been fixed by
binutils upstream. And this workaround could be dropped.
https://sourceware.org/bugzilla/show_bug.cgi?id=18548
0007-Revert-rules-remove-firmware-loading-rules.patch
0008-Revert-udev-remove-userspace-firmware-loading-suppor.patch
These two patches are dropped because they are for kernel < 3.7.
But the current minimal requirement of kernel to build systemd
is as below.
REQUIREMENTS:
Linux kernel >= 3.13
Linux kernel >= 4.2 for unified cgroup hierarchy support
So these two patches no long make any sense.
Also remove non-exist firmware-path option
0009-remove-duplicate-include-uchar.h.patch
0010-check-for-uchar.h-in-meson.build.patch
These two patches are dropped because musl has implemented
uchar.h. See commit below from musl repo.
""" ab9672ae73248f51e30f4553c4b8878525e46383
implement uchar.h (C11 UTF-16/32 conversion) interfaces
"""
0018-check-for-missing-canonicalize_file_name.patch
The above patch is dropped because current systemd does not need
canonicalize_file_name.
0025-Define-_PATH_WTMPX-and-_PATH_UTMPX-if-not-defined.patch
The above patch is dropped because utmp makes no sense in musl.
Check code below from musl.
include/utmp.h:#define _PATH_UTMP "/dev/null/utmp"
And utmp PACKAGECONFIG has been explicitly disabled for musl.
So we don't need this patch.
0032-memfd.patch
0033-basic-macros-rename-noreturn-into-_noreturn_-8456.patch
libmount.patch
0034-Fix-format-truncation-compile-failure-by-typecasting.patch
The above patches are dropped because they are backported patches.
And current systemd has contained these patches.
0036-time-util-fix-build-with-gcc8-Werror-format-truncati.patch
The above patch is dropped because it has been merged and is now
in new version.
* Newly Added Patch
0005-include-gshadow-only-if-ENABLE_GSHADOW-is-1.patch
This patch is added to fix build for musl.
0019-Do-not-disable-buffering-when-writing-to-oom_score_a.patch
This patch is added to fix the following error which caused system
unable to boot up.
systemd-udevd.service: Failed to adjust OOM setting: Invalid argument
dbus.service: Failed to adjust OOM setting: Invalid argument
0020-distinguish-XSI-compliant-strerror_r-from-GNU-specif.patch
This patch is added to fix segment fault error on musl systems.
0001-login-use-parse_uid-when-unmounting-user-runtime-dir.patch
This patch is backported to solve a race condition between
user-runtime-dir@xxx.service stop and user deletion.
2. PACKAGECONFIG and Dependency Changes
* Add a new PACKAGECONFIG item 'gshadow'. Enable it by default for glibc
but disable it by default for musl. This is because musl does not provide
gshadow.h.
* Add a new PACKAGECONFIG item 'portabled'. Default to disable it because
it's still experimental, according to the announcement letter.
"""
Currently, the support is still experimental, but this is expected to
change soon. Reflecting this experimental state, the "portablectl" binary
is not installed into /usr/bin yet.
"""
* Change 'kmod' from a hard dependency to a PACKAGECONFIG item. Default
to enable it.
* Change 'acl' from a hard dependency to a PACKAGECONFIG item. Default
to enable it.
* Remove 'readline' from DEPENDS. systemd does not need it.
* Remove 'libcgroup' from DEPENDS. The dependency on libcgroup has been
removed from systemd a long time ago. We now remove this unnecessary
dependency from DEPENDS.
3. update-alternatives changes
The utilities like shutdown, poweroff, etc. are now created as symlinks
at do_install. So there's no need to use update-alternatives mechanism
anymore to create the symlinks now. In addtion, I don't think we now
support multiple init systems at one running system, so there's really
no need to use update-alternatives mechanism here.
Also update the FILES_${PN} to include these files to avoid QA issue.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Fri, 13 Jul 2018 21:12:25 +0000 (14:12 -0700)]
curl: add PACKAGECONFIG options for brotli, built-in manpages, etc
- Add PACKAGECONFIG option for brotli (disable by default)
- Add PACKAGECONFIG option for built-in manpages (disabled by
default). Embedding a copy of the manpages within the curl binary
adds approx 60k of gzipped data and duplicates the contents of the
curl-doc package.
- Add PACKAGECONFIG option for verbose error messages (enabled by
default)
- Disable legacy NTLM http authentication via delegation to the
external winbind ntlm_auth helper (which isn't going to work
without a runtime dependency on samba).
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Mon, 16 Jul 2018 15:27:47 +0000 (16:27 +0100)]
atk: add explicit gettext-native dependency
In native builds we attempt to turn off i18n but this doesn't work in Meson
yet[1] but as the gettext class reduces dependencies in native builds we need to
add gettext-native to DEPENDS explicitly.
When wayland and egl both are on then it enables the wayland
plugin which actually fails to build because its not finding wayland-protocol
xml templates in proper location which is recipe sysroot since
we are cross compiling
Fixes build errors e.g.
make[4]: *** No rule to make target `//usr/share/wayland-protocols/unstable/linux-dmabuf/linux-dmabuf-unstable-v1.xml', needed by `drivers/dri2/linux-dmabuf-unstable-v1-client-protocol.h'. Stop.
make[4]: *** Waiting for unfinished jobs....
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Rui Wang [Mon, 16 Jul 2018 02:55:08 +0000 (10:55 +0800)]
bash: fix wrong exit status offset
In Linux，8 bits of the return code and 8 bits of the number of
the killing signal are mixed into a single value on the exit code,
so the exit status offset should be 8. But the autoconf checker
can not determine it while cross compiling, and then it is set to
the default value 0, which will cause generating the wrong exit
code if program exit with an error code.
Signed-off-by: Rui Wang <rui.wang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Mon, 16 Jul 2018 12:12:38 +0000 (13:12 +0100)]
oeqa/runtime/python: clean up Python test
Currently this is three test cases:
1) test_python_exists. Fail if python3 isn't in PATH.
2) test_python_stdout. Run a Python script and check the output is as expected
3) test_python_testfile. Check that a file test_python_stdout wrote to exists.
(1) should be a setup and skip the test module if it isn't present.
(2) and (3) should be merged, there's no point copying over a two line Python
file, and the test doesn't verify that the file doesn't exist in the first
place.
Rewrite the test to check that Python is present in a class setup so the entire
test is skipped if it isn't and do some simple rot13 to verify that bytecode is
being executed correctly.
Signed-off-by: Ross Burton <ross.burton@intel.com>
gtk-play example assumes that GTK's GL support always works if it is
compiled in and first tries to use gtkglsink element which leads to
problems if there are errors while instantiating GL context at a much
later stage. This failure isn't detected and there is no fallback at
this point.
gtkglsink works on wayland though using poky defaults with gtk+3 since
the gdk backend does support egl with wayland but it doesn't with x11.
Disable the gtk plugin to ensure gtk-play just moves on when gtkglsink
isn't present. This was the default when gtk was part of -plugins-bad
before upgrade to 1.14.0.
Fixes [YOCTO #12832]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* Bump VA-API version to 1.2.0
* Add support for hevc range extension decoding
* Add support for fast intra prediction in HEVC FEI
* Add 10/12-bit YUV render target formats
* Add fourcc code for Y210/Y216/Y410/Y416/RGB565/BGR565
* Add VA_STATUS_ERROR_NOT_ENOUGH_BUFFER
* Add VA_SURFACE_ATTRIB_USAGE_HINT_EXPORT
* Improve documentation
Tested with Intel vaapi driver. There is support for meson available but
its not being packaged as part of release tarball yet.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Andre McCurdy [Wed, 4 Jul 2018 00:55:22 +0000 (17:55 -0700)]
python-setuptools.inc: avoid using += with an over-ride
An over-ride replaces the original value regardless of whether or
not it's set up with +=. As replacing the original value seems to be
the intention here, drop the += to make it more explicit. Also some
minor recipe formatting tweaks.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Thu, 12 Jul 2018 13:44:06 +0000 (14:44 +0100)]
insane: optimise buildpath search
Instead of decoding every file we open as UTF-8 (with many errors as machine
code isn't UTF-8), convert the build path to the UTF-8 byte representation and
search for that instead.
Signed-off-by: Ross Burton <ross.burton@intel.com>
wic/engine: use up all free space when expanding partitions
Currently we just divide up the free space by the number of partitions
that need to be re-sized. This leads to problems when a user has
explicitly specified a subset of partitions (but not all) that need
to re-sized along with the sizes. As an example, for an image with 3
partitions, if we use:
wic write image.wic /dev/sdb --expand 1:10G
This would lead to paritions 2 and 3 each being re-sized to one thirds
of the free space instead of half.
Change the behavior to use up all the free space.
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
The UEFI spec implies that GPT partitions should be assumed to be on a 2048
sector boundary (for a 512 byte sector) and the current logic just
divides the free sectors available by the number of partitions that need
re-sizing, which may or may not align and the final result might
overshoot the limits imposed after alignment.
Since we are expanding already aligned partitions, just divide up the
free space in multiples of 2048. Also use the exec_cmd wrapper instead
of the subprocess call directly.
Fixes [YOCTO #12840]
Signed-off-by: Anuj Mittal <anuj.mittal@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Marek Vasut [Wed, 11 Jul 2018 15:03:08 +0000 (17:03 +0200)]
qemux86*: Add U-Boot machine configuration
Add U-Boot machine configuration for the qemux86 and qemux86-64
to allow building U-Boot on those targets. This in turn allows
the auto-updater to update the U-Boot recipe.
Signed-off-by: Marek Vasut <marex@denx.de> Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com> Cc: Otavio Salvador <otavio@ossystems.com.br> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Cc: Ross Burton <ross.burton@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Marek Vasut [Wed, 11 Jul 2018 15:03:07 +0000 (17:03 +0200)]
u-boot: Upgrade to 2018.07 release
This upgrades the U-Boot from 2018.05 to 2018.07 release.
Signed-off-by: Marek Vasut <marex@denx.de> Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com> Cc: Otavio Salvador <otavio@ossystems.com.br> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Cc: Ross Burton <ross.burton@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Marek Vasut [Wed, 11 Jul 2018 15:03:06 +0000 (17:03 +0200)]
u-boot: Add missing dependency on flex and bison native
The U-Boot build, in particular the Kconfig, depends on bison and
flex native. Add the missing dependencies.
Signed-off-by: Marek Vasut <marex@denx.de> Cc: Alexander Kanavin <alexander.kanavin@linux.intel.com> Cc: Otavio Salvador <otavio@ossystems.com.br> Cc: Richard Purdie <richard.purdie@linuxfoundation.org> Cc: Ross Burton <ross.burton@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 11 Jul 2018 14:56:51 +0000 (16:56 +0200)]
classes/buildhistory: properly process escaped chars from pkgdata
All values written out to pkgdata are escaped (see write_if_exists() in
package.bbclass). In practice there tend not to be characters that need
escaping except in the scriptlets (pkg_preinst, pkg_postinst, pkg_prerm
and pkg_postrm) where currently we still see the escape codes in the
corresponding files within buildhistory (e.g. \n and \t) and thus also
in the output of buildhistory-diff, hindering proper diffing of changes.
To fix this, when we read values from pkgdata and write them out to
buildhistory, we need to interpret the escape codes by doing the exact
reverse of what we do in package.bbclass.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 11 Jul 2018 14:56:50 +0000 (16:56 +0200)]
classes/buildhistory: handle packaged files with names containing spaces
The FILELIST field of the package info file in the buildhistory
repository is a space-separated list of all of the files in the package.
If a name of a file packaged by a recipe contains a space character then
of course the result was that we didn't handle its name properly. To fix
that, use quotes around any filename containing spaces and at the other
end use these quotes to extract the proper entries.
Fixes [YOCTO #12742].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 11 Jul 2018 14:56:49 +0000 (16:56 +0200)]
lib/oe/buildhistory_analysis: drop related field feature
The original idea here was that changes to certain fields might be able
to be explained if there was a change to another field, for example if
RDEPENDS changed it might be because DEPENDS changed. Thus we were
printing this kind of thing out with each change. Unfortunately in
practice this turned out to be noisy and not particularly useful, so we
might as well remove it.
Fixes [YOCTO #7336].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Richard Purdie [Sun, 15 Jul 2018 12:02:50 +0000 (12:02 +0000)]
oeqa/decorator: Improve reliability
Checking if the dependency had any failure is unreliable, for example
if the underlying data doesn't get transferred and the list is empty,
success of the dependency is assumed.
Since we now have success data available, change the code to use it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 13 Jul 2018 12:13:10 +0000 (13:13 +0100)]
oeqa: Remove xmlrunner
This isn't present on modern distros by default and doesn't work with
testtools, needing multiple code paths in the code. Remove it in favour
of finding a better replacement for results collection/analysis.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 12 Jul 2018 11:10:38 +0000 (11:10 +0000)]
oeqa/core/threaded: Remove in favour of using concurrenttests
We have several options for parallel processing in oeqa, parallel
execution of modules, threading and mulitple processes for the runners.
After much experimentation is appears the most scalable and least
invasive approach is multiple processes using concurrenttestsuite
from testtools. This means we can drop the current threading code
which is only used by the sdk test execution.
Richard Purdie [Mon, 9 Jul 2018 15:20:34 +0000 (15:20 +0000)]
oeqa: Add selftest parallelisation support
This allows oe-selftest to take a -j option which specifies how much test
parallelisation to use. Currently this is "module" based with each module
being split and run in a separate build directory. Further splitting could
be done but this seems a good compromise between test setup and parallelism.
You need python-testtools and python-subunit installed to use this but only
when the -j option is specified.
See notes posted to the openedmbedded-architecture list for more details
about the design choices here.
Some of this functionality may make more sense in the oeqa core ultimately.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 13 Jul 2018 21:59:22 +0000 (21:59 +0000)]
devtool: Split tests into multiple classes
This allows better parallelism between the different tests as currently
this block takes the longest time to execute. devtool tests are still
all grouped into the "devtool" module for ease of exection.
This also makes it easier to execute some subset of devtool tests for
testing devtool changes.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 14 Jul 2018 14:50:48 +0000 (14:50 +0000)]
oeqa: Default to buffer mode for tests
Currently some tests run in buffer mode and some don't. Those that don't can
corrupt stdout/stderr. Switch to using buffer mode everywhere so we're consistent.
If there is useful output on stdout/stderr, it will be displayed if the test
fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 14 Jul 2018 11:02:07 +0000 (11:02 +0000)]
oeqa/selftest/wic: Ensure initramfs exists for test_iso_image
AssertionError: Command 'wic create mkhybridiso --image-name core-image-minimal -o /var/tmp/wic.oe-selftest/' returned non-zero exit status 1:
ERROR: _exec_cmd: gzip -f -9 -c /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio > /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio.gz returned '1' instead of 0
output: gzip: /var/tmp/wic.oe-selftest/tmp.wic.drhn6edm/initrd.cpio: No such file or directory
This is because in a clean build directory, the initramfs may not be rebuilt.
Add a call to ensure it is built to avoid the error.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 13 Jul 2018 16:22:29 +0000 (16:22 +0000)]
oeqa/selftest/package: Use setUpLocal, not setUp
The main setUp function needs to be called and tests are meant to
define setUpLocal. For some reason this one didn't leading to errors
with recent code changes. Fix it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 11 Jul 2018 11:46:01 +0000 (11:46 +0000)]
oeqa/runner: Ensure we don't print misleading results output
The current code assumes if something isn't a failure of some
kind, it was a pass. When test case IDs weren't matching, this lead
to very confusing output where things would fail, then be listed as
passing.
This adds code to track successes, ensuring we don't end up in this
position again with unmatched entries being listed as UNKNOWN.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
