Ross Burton [Mon, 3 Dec 2018 11:47:58 +0000 (11:47 +0000)]
oeqa/oelib/path: don't leak temporary directories
setUp() is used to populate a directory of temporary files, and deleted in
__del__. However setUp() is called once *per test* so __del__ would only be
able to remove the last directory created.
Fix the code by using the natural counterpart to setUp, tearDown(), to clean up.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Khem Raj [Sat, 1 Dec 2018 18:12:53 +0000 (10:12 -0800)]
nfs-utils: Upgrade to 2.3.3
enhance the musl-only patch to degrade certain Werros into Warnings
which helps in compiling on musl, since its checking for __GLIBC__
defines which are undefined on musl.
fix build on x32 ABI
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Thu, 29 Nov 2018 17:39:33 +0000 (17:39 +0000)]
oeqa/sdk/python: add Python 2 and fix detection
Add a Python 2 form to exercise that if present, and fix the setUp() so it
actually looks for a package that exists (nativesdk-python3 is a virtual
package, the interpretter is in nativesdk-python3-core).
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Joshua Watt [Thu, 29 Nov 2018 14:57:20 +0000 (08:57 -0600)]
classes/update-alternatives: Skip alternatives when disabled
Skips the update alternative steps for recipes that shouldn't have them
enabled.
Fixes errors like:
nativesdk-bzip2-1.0.6-r5 do_package: bzip2: alternative target
(/opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2
or
/opt/poky/2.5+snapshot/sysroots/i686-pokysdk-mingw32/usr/bin/bunzip2.bzip2)
does not exist, skipping...
When building mingw SDKs
[YOCTO #12962]
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Douglas Royds [Wed, 28 Nov 2018 21:13:18 +0000 (10:13 +1300)]
acl: Strip build host tool details from installed ptest file
Adopt the PTEST_BUILD_HOST_FILES mechanism to strip build host paths from the
installed builddefs file. Also strip the record of the build host rpm version.
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Douglas Royds [Wed, 28 Nov 2018 04:53:10 +0000 (17:53 +1300)]
ptest: Optionally strip build host tool info from installed files
Several packages (eg. acl, attr, openssl) install Makefiles or other build host
configuration files for ptest so that they can run `make check`, for instance.
These build host files commonly include paths and versions of build host tools,
(eg. rpm or perl) whose presence and version depends on the build host distro.
Optionally strip lines from installed PTEST_BUILD_HOST_FILES using an
extended regex pattern defined in PTEST_BUILD_HOST_PATTERN. This has no
effect on the on-target ptest.
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Douglas Royds [Wed, 28 Nov 2018 04:53:09 +0000 (17:53 +1300)]
ptest: Reproducibility: Strip build host paths from any installed Makefile
A common pattern for ptest is to install a Makefile that provides a make check
target. These generated Makefiles are normally full of build host paths, to
hosttools, and to files and scripts within the source or build tree. They also
commonly include the CFLAGS and CPPFLAGS. None of these build host paths
can possibly work on-target, so stripping them has no effect on the success of
the resulting ptests.
Stripping the HOSTTOOLS_DIR has this effect, for instance:
Paul Eggleton [Thu, 29 Nov 2018 23:38:28 +0000 (12:38 +1300)]
oe-selftest: add some tests for recipeutils module
Add some tests for functions in meta/lib/oe/recipeutils.py, in
particular for a few issues I've just fixed. I haven't added tests for
all of the functions - some of them are already being tested via devtool
in any case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The code here wasn't correctly getting the variable history for
varflags, so for example if you did a devtool upgrade on a recipe where
the SRC_URI checksums were in the .inc file (typical for python recipes
in order to support both python 2 and 3) then after the upgrade the
new values would be set in the recipe and the old values were left in
the .inc, which is not right. Teach the code here how to get the history
for varflags so it works properly.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Eggleton [Wed, 28 Nov 2018 04:16:14 +0000 (17:16 +1300)]
lib/oe/recipeutils: patch_recipe(): fix handling of values across includes/classes
If we were setting a variable and part of the variable's value was being
set in a class or a .inc file, we were still just setting the value
outright on the assumption that it was too hard to do otherwise. With
some careful use of the variable history we can do better for certain
situations i.e. when the recipe does not currently set the value
outright.
Additionally, correctly remove _appends for variables we are changing if
we're trying to remove the value added in the _append.
Fixes [YOCTO #12623] and partially fixes [YOCTO #9360].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The value to SRC_URI_append_class-native was not prefixed with a space.
This was not noticed as the SRC_URI before applying the _append contains
trailing spaces. However, if one, e.g., has a .bbappend and adds to the
SRC_URI using SRC_URI += "file://foo.patch", then there no longer is any
trailing space and the _append concatenates the two URIs together,
leading to a build failue.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Anuj Mittal [Fri, 30 Nov 2018 07:16:09 +0000 (15:16 +0800)]
gstreamer1.0-vaapi: downgrade vaapisink to marginal rank
Using vaapisink (which doesn't supports DRI3 [1] and uses DRI2) with
default poky configuration currently results in an unresponsive display
because DRI2 rendering doesn't work (as of xserver 1.20.3) in non-composited
environments [2].
Downgrade vaapisink to marginal for now so playbin (and in turn gst-play
and gtk-play examples) uses next best sink element and works out of box.
While there is a bit of documentation regarding building a new
manifest file for python, it seems that users usually only read
the manifest file.
The manifest file is in JSON format which doesn't allow comments,
hence why instructions were initially put elsewhere.
This patch hacks the call to open the JSON manifest file by using a
marker to trick it into reading only part of the file as the manifest
itself, and keep the other part as comments, which contain instructions
for the user to run the create_manifest task after an upgrade or
when adding a new package.
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
While there is a bit of documentation regarding building a new
manifest file for python, it seems that users usually only read
the manifest file.
The manifest file is in JSON format which doesn't allow comments,
hence why instructions were initially put elsewhere.
This patch hacks the call to open the JSON manifest file by using a
marker to trick it into reading only part of the file as the manifest
itself, and keep the other part as comments, which contain instructions
for the user to run the create_manifest task after an upgrade or when
adding a new package.
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandr@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Thu, 29 Nov 2018 11:42:15 +0000 (11:42 +0000)]
image_types: use cpio-native to build cpio images
As per the previous commit, upstream cpio has a bug which means it crashes on
append. If the image being built has already had testimage ran then cpio-native
will be in the sysroot. It's also possible that some distributions are shipping
this broken CVE patch too.
Now that our cpio-native is fixed, until we can be sure that the host cpio isn't
broken depend on cpio-native if building a cpio image.
[ YOCTO #13042 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Thu, 29 Nov 2018 11:42:14 +0000 (11:42 +0000)]
cpio: fix crash when appending to archives
The upstream fix for CVE-2016-2037 introduced a read from uninitialized memory
bug when appending to an existing archive, which is an operation we perform when
building an image.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 29 Nov 2018 12:07:53 +0000 (12:07 +0000)]
oeqa/selftest/context: Improve log file handling
The existing logfile is simply placed in the current directory. Since the test
changes cwd to BUILDDIR, the symlink to the log can be placed in an invalid
directory. We also see trackbacks if the symlink is invalid.
Improve things by:
* Placing logs in LOG_DIR (or BUILDDIR if unset).
* Using a full path to the log meaning the log and link are placed in the same directory.
* Using lexists instead of exists so invalid symlinks are handled correctly.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 29 Nov 2018 10:40:58 +0000 (10:40 +0000)]
oeqa/selftest/buildoptions: Ensure diskmon tests run consistently
Heartbeat events default to once a second and we need to ensure we have
enough time in the task to see them.
Add a nostamp delay task 5s long so we can have a consistently timed
task which doesn't need cleanup or have unneeded dependencies. This
ensures we should deterministically see the disk moinitor events
regardless of the state of the build. This is done in a way which
doesn't corrupt build state or need cleanup and is efficient.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Fri, 16 Nov 2018 07:09:59 +0000 (23:09 -0800)]
nspr: improve reproducibility
Remove _BUILD_STRING and _BUILD_TIME in _pl_bld.h
to avoid introducing timestamp in the .so library
such as libnspr4.so as below:
$ readelf --wide --decompress --hex-dump=.rodata libnspr4.so
[snip]
0x00004000 323031382d31312d31352030353a3439 2018-11-15 05:49
[snip]
[YOCTO #12639]
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 28 Nov 2018 17:31:39 +0000 (17:31 +0000)]
scripts/runqemu: Improve lockfile handling for python with close_fd=True
On python versions with close_fds=True (python 3.2 onwards), the tap
device lockfile isn't passed to the child process.
Since this guards against use of an active interface, we really want this
here, so pass it in pass_fds. This means if the parent exits early, the child
still holds the lock, avoiding messages like:
runqemu - ERROR - Failed to run qemu: qemu-system-x86_64: could not configure /dev/net/tun (tap0): Device or resource busy
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 28 Nov 2018 17:30:10 +0000 (17:30 +0000)]
scripts/runqemu: Tidy up lock handling code
Various tweaks:
- Balance up the aquire/release functions
- Use debug messge for both acquiring and release message for consistency in logs
- Use None instead of an empty string
- Reset the value of the field if we don't have the lock any more
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The reason are the misplaced return statements in runqemu, its skipping a block
of logic when MACHINE is set in the environment when it shouldn't. Fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 27 Nov 2018 23:38:44 +0000 (23:38 +0000)]
oeqa/utils/commands: Add extra qemu failure logging
Rather than just referring the user to the logs containing the failure, print
them on the console. This aids debugging with oe-selftest with parallelisation
as the logs may otherwise be lost.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Mon, 26 Nov 2018 03:14:54 +0000 (22:14 -0500)]
multilib_script: fix packages split
In multilib_script.bbclass it renames script file which listed in
MULTILIB_SCRIPTS. It may mix up packages split. Take package curl as
example, ${bindir}/curl-config is packaged to curl-dev originally. But
it is renamed to curl-config-${MULTILIB_SUFFIX} and packaged to curl
when multilib is enabled.
Adrian Freihofer [Sun, 25 Nov 2018 19:45:03 +0000 (20:45 +0100)]
systemd: fix PN-container package splitting
- Fix package splitting for services installed with
systemd-container. Some files have been installed with systemd
package instead of systemd-container package. (*.service,
*.socket, *.html...)
- Split journal-upload, journal-remote, journal-gatewayd into
separate packages. These services are not strictly related to
nspawn containers. Remote logging is now possible without
installing all the container stuff.
- systemd-container package rrecommends journal-upload, journal-
remote, journal-gatewayd packages.
Signed-off-by: Adrian Freihofer <adrian.freihofer@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 27 Nov 2018 12:19:39 +0000 (12:19 +0000)]
oeqa/selftest/buildoptions: Improve ccache test failure output
The current failure mode doesn't show us what the logs actually looked like
and later cleans can lose them. Show the whole log in case of failure
to aid debugging intermittent problems on the autobuilder.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 27 Nov 2018 12:03:50 +0000 (12:03 +0000)]
oeqa/selftest/case: Use bb.utils.remove() instead of shutil.remove()
This avoids problems where shutil.remove will error with:
File "/usr/lib/python3.5/shutil.py", line 436, in _rmtree_safe_fd
os.unlink(name, dir_fd=topfd)
FileNotFoundError: [Errno 2] No such file or directory: 'S.gpg-agent.extra'
when there are races over file deletion (gpg agent may be slow to exit).
We already worked around speed and race issues in bb.utils.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 24 Nov 2018 17:56:06 +0000 (17:56 +0000)]
oeqa/selftest/signing: Skip tests if gpg isn't found
Raising an assertionError in the class setup isn't a particuarly good way to
indicate gpg isn't installed. Instead skip the tests if the required binary
isn't present. For the signing tests we do require it to be present and can't
use a prebuilt one.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 23 Nov 2018 15:28:28 +0000 (15:28 +0000)]
meson: Disable rpath stripping at install time
As discussed in https://github.com/mesonbuild/meson/issues/2567 there
needs to be a way to allow our rpath options passed to the linker to be
preserved, else we run into weird build failures.
(e.g. libmodulemd-native used by libdnf can't find libyaml)
Disable this for now until upstream come up with a better way of handling
this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Wed, 14 Nov 2018 09:38:16 +0000 (17:38 +0800)]
python: add tk support
Add support to enable tk via PACKGECONFIG.
before the patch:
# python
Python 2.7.15 (default, Nov 8 2018, 04:53:50)
[GCC 8.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Tkinter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python2.7/lib-tk/Tkinter.py", line 39, in <module>
import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named _tkinter
>>>
After the patch, if enable tk in PACKGECONFIG, then
# python
Python 2.7.15 (default, Oct 25 2018, 08:12:45)
[GCC 8.2.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import Tkinter
>>>
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Wed, 14 Nov 2018 09:36:11 +0000 (17:36 +0800)]
python3: add tk support
Add support to enable tk via PACKAGECONFIG.
before this patch:
# python3
Python 3.5.6 (default, Nov 8 2018, 04:53:45)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/tkinter/__init__.py", line 35, in <module>
import _tkinter # If this fails your Python may not be configured for Tk
ImportError: No module named '_tkinter'
>>>
After this patch, if enable tk in PACKAGECONFIG, then
# python3
Python 3.5.6 (default, Nov 8 2018, 03:15:52)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import tkinter
>>>
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Fri, 23 Nov 2018 02:16:31 +0000 (10:16 +0800)]
busybox: ship a symlink farm for ptest
Ship a symlink farm for busybox, which correctly considers SUID
split. This ensures that all utilities used in busybox's test cases
will first use that ones that are provided by busybox.
Modify run-ptest to prepend the directory to PATH, and also change
variable name from current_dir to current_path, as the former is
a little misleading. `readlink -f $0' gets a path to the current
script instead of the current directory.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Fri, 23 Nov 2018 02:16:24 +0000 (10:16 +0800)]
busybox: enable bzip2 by default
bunzip2 is enabled by default, but bzip2 is not. This is kind of
strange, and it also causes busybox's ptest failure regarding
bunzip2, as bunzip2's test case needs bzip2 command.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
There are no package tasks for quilt-native, so the first 4 lines doesn't
make any sense, but the glob pattern "sstate-cache/*/*" is very time
consuming when there are no disk caches. E.g., I have more than 600,000
sstate files:
- Without disk caches
# echo 3 >/proc/sys/vm/drop_caches
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
real 4m32.583s
user 0m5.768s
sys 0m12.892s
- With disk caches (e.g., run it in the second time)
$ time python3 -c 'import glob; glob.glob("/sstate-cache/*/*/sstate:quilt-native:x86_64-linux:0.65:r0:x86_64:3:*_package.tgz*")'
real 0m5.128s
user 0m2.772s
sys 0m2.308s
So the 4 removing *package* commands cost more than 20s or 272s in theory.
Robert Yang [Fri, 23 Nov 2018 10:23:00 +0000 (18:23 +0800)]
bugzilla.bbclass: Remove it since obsoleted
It is a still python2 bbclass, so it has been broken since bitbake changed to
python3 which was 2 years ago. No one reported/fixed it for python3 in recent 2
years. So we can assume that no one uses it anymore.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongxu Jia [Fri, 23 Nov 2018 07:47:20 +0000 (15:47 +0800)]
elfutils: 0.174 -> 0.175
- Drop backport CVE patches
0001-libdwfl-Sanity-check-partial-core-file-data-reads.patch
0001-size-Handle-recursive-ELF-ar-files.patch
0001-arlib-Check-that-sh_entsize-isn-t-zero.patch
- Drop patches that upstream has fixed
0005-fix-a-stack-usage-warning.patch [9a74c19 backends: ppc use define
instead of const for size of dwarf_regs array.]
- Update debian patches to 0.175
- Rebase local patch to 0.175
0008-build-Provide-alternatives-for-glibc-assumptions-hel.patch
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
