Replace usage of os.utils.getstatusoutput() with direct subprocess
calls. Pass a modified environment and working directory where necessary
to bypass the need to execute in a shell.
Signed-off-by: Joshua Watt <JPEWhacker@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Bruce Ashfield [Fri, 24 Aug 2018 14:59:49 +0000 (10:59 -0400)]
linux-yocto/standard: enable bpf configs in standard kernel
Enabling the bpf feature for the standard kernel configurations
by default. Systemd (and other) userspace applications are looking
for BPF more often, and we get warnings such as this:
[ 12.810554] systemd[1]: File
/lib/systemd/system/systemd-journald.service:36 configures an IP
firewall (IPAddressDeny=any), but the local system does not support
BPF/cgroup based firewalling.
The configs don't add much overhead to the kernel, so we enable them
by default.
Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongxu Jia [Fri, 24 Aug 2018 08:54:04 +0000 (16:54 +0800)]
terminal.bbclass: use var-SHELL as the shebang of wrapper script
The devshell.bbclass set var-SHELL to var-DEVSHELL, and terminal.bbclass
initial var-SHELL with `bash'. Keep sync with it, use var-SHELL rather
than hardcoded `/bin/sh' as the shebang of wrapper script.
On Ubuntu host, default shell is dash (/bin/sh -> dash), even though
we assign var-SHELL with `/bin/bash', the wrapper script is still dashism.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mike Looijmans [Fri, 24 Aug 2018 07:21:44 +0000 (09:21 +0200)]
busybox/mdev-mount.sh: Fix partition detect and cleanup mountpoint on fail
This fixes issues mainly seen when mounting eMMC devices:
The wildcard /sys/block/${DEVBASE}/${DEVBASE}*1 matches both "mmcblk0p1"
and "mmcblk0boot1" for example, and this results in syntax errors. Fix this
by searching for a "partition" file instead, which only exists for real
partitions and not 'fakes' like the eMMC extra's.
When mount fails, the mountpoint file is left behind, causing later attempts
at auto-mounting it to fail. If mount fails, remove the mountpoint, leaving
the system in the state as it was before the mount attempt.
Signed-off-by: Mike Looijmans <mike.looijmans@topic.nl> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yi Zhao [Fri, 24 Aug 2018 07:21:27 +0000 (15:21 +0800)]
blktrace: Security fix CVE-2018-10689
CVE-2018-10689: blktrace (aka Block IO Tracing) 1.2.0, as used with the
Linux kernel and Android, has a buffer overflow in the dev_map_read
function in btt/devmap.c because the device and devno arrays are too
small, as demonstrated by an invalid free when using the btt program
with a crafted file.
Hongxu Jia [Fri, 24 Aug 2018 07:00:31 +0000 (15:00 +0800)]
package.bbclass: only one hardlink of separated debug info file in each directory
While multiple hardlinks of binary located in different dirs,
there are also multiple hardlinks of separated debug info file
with the same binary name in same debug dirs. But in each dir,
only one debug file with original name works. Because all of
binary hardlinks have one `.gnu_debuglink' which is added in
`splitdebuginfo'. It caused gdb could not find debugging
symbols.
Mingli Yu [Fri, 17 Aug 2018 06:27:10 +0000 (14:27 +0800)]
mingetty: fix usrmerge install path
Update /sbin to $base_sbindir to fix the
below warning when usrmerge enabled in
DISTRO_FEATURES.
WARNING: mingetty-1.08-r3 do_package: mingetty: NOT adding alternative provide /usr/sbin/getty: /usr/sbin/mingetty does not exist
WARNING: mingetty-1.08-r3 do_package: QA Issue: mingetty: Files/directories were installed but not shipped in any package:
/sbin
/usr/sbin
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 23 Aug 2018 16:32:17 +0000 (17:32 +0100)]
scripts: Add autobuilder worker test script
Add a script which can be run on new autobuilder workers to check all needed configuration
is present. Designed to be run in a repo where bitbake/oe-core are already present.
This means when we add new autobuilder workers, we can quickly test whether all the needed
funcationality to support the standard yocto project autobuilder (ie. the standard
OE-Core tests) are present.
It uses images prebuilt in a previous release to cut build/testing time.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Thu, 23 Aug 2018 01:51:31 +0000 (18:51 -0700)]
openssl: remove dependency on relative_symlinks class
Although the relative_symlinks class converts any absolute symlinks
in ${D} into relative symlinks automatically, it's a little clearer
to create relative symlinks directly where possible.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Thu, 23 Aug 2018 08:07:25 +0000 (16:07 +0800)]
lib/oe/patch.py: Clean up getstatusoutput usage
We can't use subprocess.check_output() or subprocess.call() here since the one
who invokes runcmd() needs handle CmdError() exception (error out or ignore
it).
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 22 Aug 2018 16:43:06 +0000 (16:43 +0000)]
lib/oe/utils: Fix get_multilib_datastore to work for original tune
Currently the original datastore returned by this function doesn't
always work as the tune isn't set back to the original. Fix it
to work like all_multilib_tune_list() in utils.bbclass and correct
the data returned.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 22 Aug 2018 14:51:17 +0000 (14:51 +0000)]
glibc: Improve ldd loader specification
Currently if a tune isn't specified in the table, the loader defaults for the
architecture are used which may or may not match our path specification. This
leads to general confusion.
Change the code to use the linuxloader class which works of architecture, not
tune.
This still isn't perfect as n32/x32 aren't covered but its an improvement
to listing all tunes here.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
double64_init: Check psf->sf.channels against upper bound
This prevents division by zero later in the code.
While the trivial case to catch this (i.e. sf.channels < 1) has already
been covered, a crafted file may report a number of channels that is
so high (i.e. > INT_MAX/sizeof(double)) that it "somehow" gets
miscalculated to zero (if this makes sense) in the determination of the
blockwidth. Since we only support a limited number of channels anyway,
make sure to check here as well.
CVE-2017-14634
Closes: #318
Affects libsndfile1 = 1.0.28
Signed-off-by: Jagadeesh Krishnanjanappa <jkrishnanjanappa@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Wed, 22 Aug 2018 00:43:30 +0000 (17:43 -0700)]
gnutls: drop PACKAGECONFIG options for SSL v3 and TLS v1.3
By including PACKAGECONFIG options, the recipe takes responsibility
for defining the default state of these options. Although the recipe
currently aligns with the gnutls defaults (ie both disabled) tracking
new gnutls releases will be a maintenance effort. Unless there's a
clear reason to do otherwise, it seems safer to leave the choice of
which SSL/TLS versions to enable by default up to the gnutls
developers.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Wed, 22 Aug 2018 00:43:29 +0000 (17:43 -0700)]
gnutls: drop obsolete configure.ac patch
>From gnutls 3.5.8 onwards, the code in configure.ac has been passing
"basename $i" to sed, rather than "echo $i". Since the full ${srcdir}
path is not being processed, there's no risk of unexpected matches.
Richard Purdie [Wed, 22 Aug 2018 12:47:10 +0000 (12:47 +0000)]
oeqa/runtime/ldd: Clean up test
* Merge the two tests together as having them separate is pointless
* Test that ldd runs correctly
* Add in a dependency on the "ldd" package being installed instead of
the sdk tools feature
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jens Rehsack [Wed, 22 Aug 2018 09:04:24 +0000 (11:04 +0200)]
libxml-parser-perl: fix "...contains bad RPATH"
The perl distribution "XML-Parser" relies for configuration
on the tooling of Devel::CheckLib - which is not aware of
sysroot locations nor of reasonable compiler/link definitions
from outside.
This causes
ERROR: libxml-parser-perl-2.44-r0 do_package_qa: QA Issue: package libxml-parser-perl contains bad RPATH ${BUILDDIR}/tmp/work/core2-64-poky-linux/libxml-parser-perl/2.44-r0/recipe-sysroot/usr/lib in file ${BUILDDIR}/tmp/work/core2-64-poky-linux/libxml-parser-perl/2.44-r0/packages-split/libxml-parser-perl/usr/lib/perl/vendor_perl/5.24.4/auto/XML/Parser/Expat/Expat.so
package libxml-parser-perl contains bad RPATH ${BUILDDIR}/tmp/work/core2-64-poky-linux/libxml-parser-perl/2.44-r0/recipe-sysroot/usr/lib in file ${BUILDDIR}/tmp/work/core2-64-poky-linux/libxml-parser-perl/2.44-r0/packages-split/libxml-parser-perl/usr/lib/perl/vendor_perl/5.24.4/auto/XML/Parser/Expat/Expat.so [rpaths]
ERROR: libxml-parser-perl-2.44-r0 do_package_qa: QA run found fatal errors. Please consider fixing them.
ERROR: libxml-parser-perl-2.44-r0 do_package_qa: Function failed: do_package_qa
It's strongly encouraged to the maintainer @toddr to rework the
toolchain for up to date environments.
[RP: Added fix for nativesdk RPATH issues too]
Signed-off-by: Jens Rehsack <sno@netbsd.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jens Rehsack [Wed, 22 Aug 2018 09:04:23 +0000 (11:04 +0200)]
cpan.bbclass: adopt to recent EU::MM
The modern the time, the improvements in ExtUtils::MakeMaker.
Nowadays, .packlist and perllocal.pod aren't touched anymore when appropriate
flags set during configure stage. Controlling the flags globally avoids
dual-life recipes need share patching.
Further: remove prepending ${PERL_ARCHLIB} in PERL5LIB - it's wrong (search
order is site_lib, vendor_lib, core) - and ${PERL_ARCHLIB} contains core
libpath only ...
Signed-off-by: Jens Rehsack <sno@netbsd.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongzhi.Song [Wed, 22 Aug 2018 08:37:12 +0000 (01:37 -0700)]
cryptodev-linux: Fixes a kernel crash observed with cipher-gcm test
The crypto API for AEAD ciphers changed in recent kernels, so that
associated data is now part of both source and destination scatter
gathers. The source, destination and associated data buffers need
to be stiched accordingly for the operations to succeed.
Signed-off-by: Hongzhi.Song <hongzhi.song@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongxu Jia [Wed, 22 Aug 2018 09:16:42 +0000 (17:16 +0800)]
mtools: fix race issue while mtools invoked frequently
While invoking mtools frequently, the unblocking request
caused race issue. Here is an example of syslinux
[snip]
dd if=/dev/zero of=floppy.img bs=1024 count=144
losetup /dev/loop1 floppy.img
mkdosfs /dev/loop1
syslinux -i /dev/loop1
|plain floppy: device "/proc/6351/fd/3" busy (Resource temporarily unavailable):
|Cannot initialize 'S:'
|Bad target s:/ldlinux.sys
[snip]
The idea is from:
https://bugzilla.redhat.com/show_bug.cgi?id=1235016
https://groups.google.com/a/chromium.org/forum/#!msg/chromium-os-dev/bRPUCFHoBTQ/ZjB8kjjx1vUJ
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-run-native: Add *-native directories under STAGING_BINDIR_NATIVE to PATH environment
It helps to find/use native tools under ${STAGING_BINDIR_NATIVE}/*-native.
Solving below error:
$ oe-run-native python3-native python3
Running bitbake -e python3-native
Error: Unable to find 'python3' in .../tmp/work/x86_64-linux/python3-native/3.5.5-r1.0/recipe-sysroot-native/usr/bin:.../tmp/work/x86_64-linux/python3-native/3.5.5-r1.0/recipe-sysroot-native/bin:.../tmp/work/x86_64-linux/python3-native/3.5.5-r1.0/recipe-sysroot-native/usr/sbin:.../tmp/work/x86_64-linux/python3-native/3.5.5-r1.0/recipe-sysroot-native/sbin
Error: Have you run 'bitbake python3-native -caddto_recipe_sysroot'?
-- snip --
After this change we have native python3 to be found:
$ oe-run-native python3-native python3
Running bitbake -e python3-native
Python 3.5.5 (default, Aug 8 2018, 17:45:49)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-28)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
-- snip --
[YOCTO #12889]
Signed-off-by: Jagadeesh Krishnanjanappa <jkrishnanjanappa@mvista.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Mon, 20 Aug 2018 08:57:54 +0000 (16:57 +0800)]
cmake-native: fix to function correctly in case of eSDK
Our eSDK is expected to provide traditional SDK's functionality. But
for cmake, it could not function well in eSDK.
This problem is discovered by the assimp.py test case. The error message
is as below.
testsdkext/tmp/sysroots/x86_64/usr/lib/libz.so: error adding symbols: file in wrong format
collect2: error: ld returned 1 exit status
The problem is about cmake-native being unable to find the correct lib.
nativesdk-cmake has solved this problem. So make use of the solution to
solve the eSDK problem.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 21 Aug 2018 21:45:31 +0000 (21:45 +0000)]
oeqa/context: Only set buffer mode for non-concurrent tests
Periodically we'd see:
NOTE: core-image-sato-1.0-r0 do_testsdk: ======================================================================
NOTE: core-image-sato-1.0-r0 do_testsdk: ERROR: broken-runner
NOTE: core-image-sato-1.0-r0 do_testsdk: ----------------------------------------------------------------------
NOTE: core-image-sato-1.0-r0 do_testsdk: testtools.testresult.real._StringException: Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-mips/build/meta/lib/oeqa/core/utils/concurrencytest.py", line 122, in _run_test
test.run(process_result)
File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 1194, in run
protocol = TestProtocolServer(result, self._passthrough, self._forward)
File "/usr/lib/python3.6/site-packages/subunit/__init__.py", line 514, in __init__
stream = stream.buffer
AttributeError: '_io.StringIO' object has no attribute 'buffer'
which seems to occur if a result arrives before all the runner threads
have started. The runner's result handling changes sys.stdout to a buffer
temporarily which can be seen in other threads and it can sometimes fail.
Since the tests are running in a separate process we don't need this buffer
handling in the concurrent case so only set when not parallelising. The
concurrent class handle setting buffer mode internally.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 21 Aug 2018 18:28:32 +0000 (18:28 +0000)]
oeqa/concurrencytest: Ensure subunit streams are flushed at exit
Without this, error output such as that in the teardown can be lost
and processes may recieve signals they're not expecting causing other
strange errors.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongxu Jia [Tue, 21 Aug 2018 05:44:55 +0000 (13:44 +0800)]
python/python3: add virtual/crypt to DEPENDS
Since `6146b8c glibc: Disable crypt support in glibc' in oe-core,
python2/3 could not find symbol crypt which caused import crypt failed.
[snip]
>>> import crypt
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib64/python3.5/crypt.py", line 3, in <module>
import _crypt
ImportError: /usr/lib64/python3.5/lib-dynload/_crypt.cpython-35m-x86_64-linux-gnu.so: undefined symbol: crypt
[snip]
Add virtual/crypt to DEPENDS, and python's build system (setup.py)
will search libcrypt.so in recipe-sysroot and add `-lcrypt' if it
exists.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
remove the indirect dependcy of autoconf-archive-native via
SSTATE_EXCLUDEDEPS_SYSROOT to avoid not needed .m4 installed
into sysroot, which may cause compile problem.
Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Sun, 19 Aug 2018 22:16:02 +0000 (22:16 +0000)]
kernel-artifact-names.bbclass: Add 2 more variables to make it easier to change all names with one variable
* some people don't like the ${MACHINE} in the symlink, because now the DEPLOYDIR already
contains ${MACHINE} subdirectory, add KERNEL_ARTIFACT_LINK_NAME variable to change it
in one place without the need to list all variables for various artifacts
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Sun, 19 Aug 2018 22:16:01 +0000 (22:16 +0000)]
kernel-artifact-names.bbclass, kernel.bbclass: remove prefix and extension from MODULE_TARBALL_* variables
* for consistency with other artifacts variables, include only the version string, not the actual name or extension
* changing .tgz to something else in the MODULE_TARBALL_NAME variable only wouldn't make much sense
because then kernel.bbclass still calls "tar -cvzf" to create it
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Sun, 19 Aug 2018 22:16:00 +0000 (22:16 +0000)]
kernel*.bbclass: rename *_SYMLINK_NAME variables to *_LINK_NAME and *_BASE_NAME to *_NAME
* for consistency with IMAGE_NAME and IMAGE_LINK_NAME
and to avoid confusion with IMAGE_BASENAME (which is the
actual name of the artifact, e.g. PN while KERNEL_IMAGE_BASE_NAME
was only the version suffix)
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Sun, 19 Aug 2018 22:15:58 +0000 (22:15 +0000)]
bitbake.conf, kernel-artifact-names.bbclass: introduce IMAGE_VERSION_SUFFIX instead of using DATETIME directly
* this makes it easier to use different version string than DATETIME, e.g. set from jenkins job
while keeping the suffix consistent across all artifacts stored in DEPLOYDIR
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Wang Quanyang [Fri, 17 Aug 2018 15:01:37 +0000 (11:01 -0400)]
weston-init: run login before start weston.service
When systemd start the weston.service, the script "weston-start" will
check if the dir "XDG_RUNTIME_DIR" (usually is /run/user/0) exits and
create it. Then weston will create a socket file "wayland-0" for communications
with clients in this dir.
If systemd is built with enabling "pam" feature, the login will call "run-user-0.mount"
to mount tmpfs at the dir "/run/user/0", then the socket file "wayland-0" will be
missing since it is created in the old "/run/user/0".
So add "PAMName=login" to let weston.service call login first, once tmpfs is mounted at
"/run/user/0", then call weston-start to create a socket file in it.
Signed-off-by: Quanyang Wang <quanyang.wang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Fri, 17 Aug 2018 04:43:36 +0000 (21:43 -0700)]
openssl: fix path in nativesdk environment-setup script
A single version of the openssl.sh environment-setup script is
currently shared by both the openssl 1.0 and 1.1 recipes. The libdir
path in the script needs to be tweaked for openssl 1.1.
Signed-off-by: Andre McCurdy <armccurdy@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Andre McCurdy [Fri, 17 Aug 2018 04:43:35 +0000 (21:43 -0700)]
openssl: move the libdir openssl.cnf symlink into the openssl package
The openssl 1.0 recipe puts the libdir symlink to /etc/ssl/openssl.cnf
in the base openssl package (along with the libdir symlinks to
/etc/ssl/certs and /etc/ssl/private). Keep the openssl 1.1 recipe
aligned with that approach until there's a clear reason to do
something else. For more background, see comments in the following
thread:
