Having image-buildinfo enabled causes containerimage.ContainerImageTests.test_expected_files
to fail due to the presence of an unexpected file:
['./',
'./etc/',
- './etc/build',
'./etc/default/',
'./etc/default/postinst',
Tweak the class to allow it to be disabled and disable it from the test just in
case it was enabled.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Randy MacLeod [Wed, 7 Nov 2018 17:59:22 +0000 (12:59 -0500)]
valgrind: drop mips n32 support
valgrind for qemumips64 multilib builds fails to configure
for libn32 with the error:
configure:6190: checking for 32 bit build support
...
fatal error: bits/long-double-32.h: No such file or directory
It seems that the toolchain is producing:
tmp-glibc/sysroots-components/mips64-n32/libn32-glibc/usr/include/bits/long-double-n32.h
Until the toolchain problem is resolved, skip valgrind for libn32.
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Play the whack-a-mole game and add the .file directive to another
assembly file that otherwise shows itself in ld-2.28.so debug file,
which in turns alters the build-id of ld-2.28.so on target.
Signed-off-by: Martin Hundebøll <martin@geanix.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Due to a bug in find [1], -ignore_readdir_race does not work correctly with
-delete. This can lead to spurious build failures when files disappear
while such a command is running; specifically this was seen in the case of
do_configure and do_populate_lic running concurrently for packages
with ${B} == ${WORKDIR}:
find: '.../sstate-build-populate_lic': No such file or directory
While the issue is fixed in the findutils git master, the find command of
the host system is called here, so we can't ensure that the used version
contains the fix. Many common distros have not updated to a recent enough
findutils version yet (Ubuntu 18.10 contains the fix, while 18.04 is still
affected).
Work around the issue by passing the output of find to 'rm -f' instead of
using -delete.
[1] https://savannah.gnu.org/bugs/?52981
Signed-off-by: Matthias Schiffer <matthias.schiffer@ew.tq-group.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alexey Brodkin [Wed, 31 Oct 2018 18:54:23 +0000 (21:54 +0300)]
perf: Disable libunwind for ARC & RISCV64
libunwind is not yet ported for ARC & RISCV64 and on attempt
to build it for those arches we just get an error message.
If we explicitly disable libunwind it is gracefully handled by
perf build system and it just gets configured to not use it
so perf is still usable even on those arches.
Signed-off-by: Alexey Brodkin <abrodkin@synopsys.com> Cc: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changqing Li [Thu, 1 Nov 2018 03:15:17 +0000 (11:15 +0800)]
apt: update SRC_URI
update SRC_URI since previous link is not valid now
Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Fabien Lahoudere [Mon, 29 Oct 2018 11:02:29 +0000 (12:02 +0100)]
archiver: Drop unwanted directories
In sources directory we can find patches/ and temp/.
The first one is filled with symbolic link unusable on another
machines.
The second contains yocto logs to create this archives and are
typically copied when 'S = "${WORKDIR}"'
Signed-off-by: Fabien Lahoudere <fabien.lahoudere@collabora.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kosta Zertsekel [Sun, 4 Nov 2018 19:24:46 +0000 (21:24 +0200)]
meta: Use double colon for chown OWNER:GROUP
Rationale - excerp from `info chown`
====================================
OWNER‘:’GROUP
If the OWNER is followed by a colon and a GROUP (a group name or
numeric group ID), with no spaces between them, the group ownership
of the files is changed as well (to GROUP).
Some older scripts may still use ‘.’ in place of the ‘:’ separator.
POSIX 1003.1-2001 (*note Standards conformance::) does not require
support for that, but for backward compatibility GNU ‘chown’ supports
‘.’ so long as no ambiguity results. New scripts should avoid the use
of ‘.’ because it is not portable, and because it has undesirable
results if the entire OWNER‘.’GROUP happens to identify a user whose
name contains ‘.’.
Signed-off-by: Kosta Zertsekel <zertsekel@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 5 Nov 2018 15:52:09 +0000 (15:52 +0000)]
oeqa/selftest: Add test for Yocto source mirror functionality/completeness
We've had a number of occasions where the Yocto Project source mirrors have not
been complete or functioning correctly. This adds a test so that if this happens
we find out out it sooner.
It also only works over http meaning we should be able to test that anyone behind
an http only proxy (no git protocol) also has functional fetches for OE-Core and
layers built by the core of the project.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Serhey Popovych [Wed, 31 Oct 2018 14:55:47 +0000 (16:55 +0200)]
v86d: Make cross compilation working on more architectures
Since commit 709c603dec19 ("v86d: Accept aarch64 as build host") we
support cross compilation on aarch64 host in addition to x86 host.
However building on hosts different than two above will fail.
Make cross compilation support more generic by checking for TARGET_ARCH
in v86d configure script with fallback to `uname -m` when not present in
environment and pass TARGET_ARCH explicitly in do_configure().
Cross build for x86 tested on IBM Power 8 machine with RHEL7. Should
work on aarch64 and rest too.
Signed-off-by: Serhey Popovych <serhe.popovych@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 30 Oct 2018 11:18:54 +0000 (11:18 +0000)]
oe-init-buildenv/base: Relax python version checks in favour of HOSTTOOLS manipulation
Several distros are now shipping "python" as python v3 contra to the original
python guidelines. This causes users confusion/pain in trying to use our tools.
We can just force "python" to "python2" within HOSTTOOLS to avoid this issue
and hide the complexity from the user.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 2 Nov 2018 13:13:43 +0000 (13:13 +0000)]
oeqa/runtime/ptest: Inject results+logs into stored json results file
This allows the ptest results from ptest-runner, run in an image to be
transferred over to the resulting json results output.
Each test is given a pass/skip/fail so individual results can be monitored
and the raw log output from the ptest-runner is also dumped into the
results json file as this means after the fact debugging becomes much easier.
Currently the log output is not split up per test but that would make a good
future enhancement.
I attempted to implement this as python subTests however it failed as the
output was too confusing, subTests don't support any kind of log
output handling, subTest successes aren't logged and it was making things
far more complex than they needed to be.
We mark ptest-runner as "EXPECTEDFAILURE" since its unlikely every ptest
will pass currently and we don't want that to fail the whole image test run.
Its assumed there would be later analysis of the json output to determine
regressions. We do have to change the test runner code so that
'unexpectedsuccess' is not a failure.
Also, the test names are manipuated to remove spaces and brackets with
"_" used as a replacement and any duplicate occurrences truncated.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
gcc-crosssdk buils leaks config.log's through "gcc-stashed-builddir" and
TARGET_* flags to libgcc cross-build through "gcc/libgcc.mvars" file
on "gcc-stashed-builddir". This means that if BUILD_CFLAGS contains
host-specific flags like "-isystem/usr/include" libgcc build will
fail "do_qa_configure" and "do_package_qa" checks.
Remove host-related flags from TARGET_* flags for gcc-crosssdk builds.
[YOCTO #11874]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This changes the PN addtion to something containing the tune rather than
the arch which avoids these kinds of errors. If go-cross can be tune
independent that would be nice but currently that isn't the case.
[YOCTO #12586]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Tue, 30 Oct 2018 06:32:42 +0000 (14:32 +0800)]
go-dep: disable PTEST_ENABLED for mips and mips64
The current go-dep does not compile ptest successfully on mips
and mips64. So as a workaround, disable PTEST_ENABLED explicitly
to avoid error like below.
| vet config not found
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hugues Kamba [Tue, 30 Oct 2018 09:37:25 +0000 (09:37 +0000)]
python3: Fix python3-pyvenv run-time dependency
Pyvenv is just a small script that uses venv to create virtual
environments.
https://www.python.org/dev/peps/pep-0405/#creating-virtual-environments
This patch adds the python3-venv module as a self-contained package which
python3-pyvenv must depend on at run-time.
The patch also provides the package python3-pyvenv from the pyhton3-venv
package.This is good for future-proofing since python3-pyvenv has been
deprecated and only python3-venv is now available in Python 3.6.
https://docs.python.org/3/library/venv.html.
Without this patch python3-pyvenv is broken because it is missing the
venv module at run-time. This patch specifies the newly created
python3-venv as a run-time dependency of python3-pyvenv.
Signed-off-by: Hugues Kamba <hugues.kamba@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
pixman: Trim license info extracted from pixman-matrix.c
Four unrelated lines were extracted from pixman-matrix.c for the
license information.
License-Update: Only extract the relevant part from pixman-matrix.c Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libgpg-error: Trim license info extracted from init.c & gpg-error.h.in
License-Update: Only extract relevant parts from init.c & gpg-error.h.in Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
apr-util: Trim license info extracted from apu_version.h
Two unrelated lines were extracted from apu_version.h for the license
information.
License-Update: Only extract the relevant part from apu_version.h Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Two unrelated lines were extracted from apr_lib.h for the license
information.
License-Update: Only extract the relevant part from apr_lib.h Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Once the value of SOURCE_DATE_EPOCH is determined, it is stored in the recipe's SDE_FILE.
If none of the existing mechanisms are suitable, replace the do_deploy_source_date_epoch task
with recipe-specific functionality to write the appropriate SOURCE_DATE_EPOCH into the SDE_FILE.
Signed-off-by: Douglas Royds <douglas.royds@taitradio.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Hongxu Jia [Mon, 5 Nov 2018 08:03:36 +0000 (16:03 +0800)]
ghostscript: fix CVE-2018-18073
Artifex Ghostscript allows attackers to bypass a sandbox protection
mechanism by leveraging exposure of system operators in the saved
execution stack in an error object.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Hongxu Jia [Mon, 5 Nov 2018 08:03:35 +0000 (16:03 +0800)]
ghostscript: fix CVE-2018-17961
Artifex Ghostscript 9.25 and earlier allows attackers to bypass a
sandbox protection mechanism via vectors involving errorhandler
setup. NOTE: this issue exists because of an incomplete fix for
CVE-2018-17183.
Signed-off-by: Hongxu Jia <hongxu.jia@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Armin Kuster [Mon, 29 Oct 2018 01:29:32 +0000 (01:29 +0000)]
tzdata: update to 2018g
Changes to code
When generating TZif files with leap seconds, zic no longer uses a
format that trips up older 32-bit clients, fixing a bug introduced
in 2018f. (Reported by Daniel Fischer.) Also, the zic workaround
for QTBUG-53071 now also works for TZif files with leap seconds.
The translator to rearguard format now rewrites the line
"Rule Japan 1948 1951 - Sep Sat>=8 25:00 0 S" to
"Rule Japan 1948 1951 - Sep Sun>=9 1:00 0 S".
This caters to zic before 2007 and to Oracle TZUpdater 2.2.0
and earlier. (Reported by Christos Zoulas.)
Changes to past time zone abbreviations
Change HDT to HWT/HPT for WWII-era abbreviations in Hawaii.
This reverts to 2011h, as the abbreviation change in 2011i was
likely inadvertent.
Changes to documentation
tzfile.5 has new sections on interoperability issues.
Armin Kuster [Sun, 28 Oct 2018 23:38:32 +0000 (23:38 +0000)]
tzcode-native: update to 2018g
Changes to code
When generating TZif files with leap seconds, zic no longer uses a
format that trips up older 32-bit clients, fixing a bug introduced
in 2018f. (Reported by Daniel Fischer.) Also, the zic workaround
for QTBUG-53071 now also works for TZif files with leap seconds.
The translator to rearguard format now rewrites the line
"Rule Japan 1948 1951 - Sep Sat>=8 25:00 0 S" to
"Rule Japan 1948 1951 - Sep Sun>=9 1:00 0 S".
This caters to zic before 2007 and to Oracle TZUpdater 2.2.0
and earlier. (Reported by Christos Zoulas.)
Changes to past time zone abbreviations
Change HDT to HWT/HPT for WWII-era abbreviations in Hawaii.
This reverts to 2011h, as the abbreviation change in 2011i was
likely inadvertent.
Changes to documentation
tzfile.5 has new sections on interoperability issues.
Ross Burton [Thu, 1 Nov 2018 11:15:58 +0000 (11:15 +0000)]
xserver-xorg: fix CVE-2018-14665
Incorrect command-line parameter validation in the Xorg X server can lead to
privilege elevation and/or arbitrary files overwrite, when the X server is
running with elevated privileges (ie when Xorg is installed with the setuid bit
set and started by a non-root user). The -modulepath argument can be used to
specify an insecure path to modules that are going to be loaded in the X server,
allowing to execute unprivileged code in the privileged process. The -logfile
argument can be used to overwrite arbitrary files in the file system, due to
incorrect checks in the parsing of the option.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Armin Kuster [Mon, 22 Oct 2018 14:05:12 +0000 (15:05 +0100)]
tzdata: update to 2018f
Briefly:
Volgograd moves from +03 to +04 on 2018-10-28.
Fiji ends DST 2019-01-13, not 2019-01-20.
Most of Chile changes DST dates, effective 2019-04-06.
Changes to future timestamps
Volgograd moves from +03 to +04 on 2018-10-28 at 02:00.
(Thanks to Alexander Fetisov and Stepan Golosunov.)
Fiji ends DST 2019-01-13 instead of the 2019-01-20 previously
predicted. (Thanks to Raymond Kumar.) Adjust future predictions
accordingly.
Most of Chile will end DST on the first Saturday in April at 24:00 mainland
time, and resume DST on the first Saturday in September at 24:00 mainland
time. The changes are effective from 2019-04-06, and do not affect the
Magallanes region modeled by America/Punta_Arenas. (Thanks to Juan Correa
and Tim Parenti.) Adjust future predictions accordingly.
Changes to past timestamps
The 2018-05-05 North Korea 30-minute time zone change took place
at 23:30 the previous day, not at 00:00 that day.
China's 1988 spring-forward transition was on April 17, not
April 10. Its DST transitions in 1986/91 were at 02:00, not 00:00.
(Thanks to P Chan.)
Fix several issues for Macau before 1992. Macau's pre-1904 LMT
was off by 10 s. Macau switched to +08 in 1904 not 1912, and
temporarily switched to +09/+10 during World War II. Macau
observed DST in 1942/79, not 1961/80, and there were several
errors for transition times and dates. (Thanks to P Chan.)
The 1948-1951 fallback transitions in Japan were at 25:00 on
September's second Saturday, not at 24:00. (Thanks to Phake Nick.)
zic turns this into 01:00 on the day after September's second
Saturday, which is the best that POSIX or C platforms can do.
Incorporate 1940-1949 Asia/Shanghai DST transitions from a 2014
paper by Li Yu, replacing more-questionable data from Shanks.
Changes to time zone abbreviations
Use "PST" and "PDT" for Philippine time. (Thanks to Paul Goyette.)
Changes to documentation
New restrictions: A Rule name must start with a character that
is neither an ASCII digit nor "-" nor "+", and an unquoted name
should not use characters in the set "!$%&'()*,/:;<=>?@[\]^`{|}~".
The latter restriction makes room for future extensions (a
possibility noted by Tom Lane).
tzfile.5 now documents what time types apply before the first and
after the last transition, if any.
Documentation now uses the spelling "timezone" for a TZ setting
that determines timestamp history, and "time zone" for a
geographic region currently sharing the same standard time.
The name "TZif" is now used for the tz binary data format.
tz-link.htm now mentions the A0 TimeZone Migration utilities.
(Thanks to Aldrin Martoq for the link.)
Changes to build procedure
New 'make' target 'rearguard_tarballs' to build the rearguard
tarball only. This is a convenience on platforms that lack lzip
if you want to build the rearguard tarball. (Problem reported by
Deborah Goldsmith.)
tzdata.zi is now more stable from release to release. (Problem
noted by Tom Lane.) It is also a bit shorter.
tzdata.zi now can contain comment lines documenting configuration
information, such as which data format was selected, which input
files were used, and how leap seconds are treated. (Problems
noted by Lester Caine and Brian Inglis.) If the Makefile defaults
are used these comment lines are absent, for backward
compatibility. A redistributor intending to alter its copy of the
files should also append "-LABEL" to the 'version' file's first
line, where "LABEL" identifies the redistributor's change. Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Armin Kuster [Mon, 22 Oct 2018 14:05:11 +0000 (15:05 +0100)]
tzcode: update to 2018f
Changes to code
zic now always generates TZif files where time type 0 is used for
timestamps before the first transition. This simplifies the
reading of TZif files and should not affect behavior of existing
TZif readers because the same set of time types is used; only
their internal indexes may have changed. This affects only the
legacy zones EST5EDT, CST6CDT, MST7MDT, PST8PDT, CET, MET, and
EET, which previously used nonzero types for these timestamps.
Because of the type 0 change, zic no longer outputs a dummy
transition at time -2**59 (before the Big Bang), as clients should
no longer need this to handle historical timestamps correctly.
This reverts a change introduced in 2013d and shrinks most TZif
files by a few bytes.
zic now supports negative time-of-day in Rule and Leap lines, e.g.,
"Rule X min max - Apr lastSun -6:00 1:00 -" means the transition
occurs at 18:00 on the Saturday before the last Sunday in April.
This behavior was documented in 2018a but the code did not
entirely match the documentation.
localtime.c no longer requires at least one time type in TZif
files that lack transitions or have a POSIX-style TZ string. This
future-proofs the code against possible future extensions to the
format that would allow TZif files with POSIX-style TZ strings and
without transitions or time types.
A read-access subscript error in localtime.c has been fixed.
It could occur only in TZif files with timecnt == 0, something that
does not happen in practice now but could happen in future versions.
localtime.c no longer ignores TZif POSIX-style TZ strings that
specify only standard time. Instead, these TZ strings now
override the default time type for timestamps after the last
transition (or for all time stamps if there are no transitions),
just as DST strings specifying DST have always done.
leapseconds.awk now outputs "#updated" and "#expires" comments,
and supports leap seconds at the ends of months other than June
and December. (Inspired by suggestions from Chris Woodbury.) Signed-off-by: Armin Kuster <akuster808@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Fri, 26 Oct 2018 09:45:36 +0000 (17:45 +0800)]
groff: not search fonts on build host
groff searches fonts on build host which are provided by ghostscript.
The number of font files installed by groff are different according to
whether ghostscript fonts are installed on build host. Fix it by not
search font dirs on the host.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Christian Meusel [Thu, 25 Oct 2018 16:48:54 +0000 (18:48 +0200)]
gitignore: Ignore repo tool directory
This reduces the noise produced by 'git status' and 'repo status' when
orchestrating the layers with Google's repo tool
(https://gerrit.googlesource.com/git-repo/).
Signed-off-by: Christian Meusel <christian.meusel@posteo.de> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 29 Oct 2018 13:46:52 +0000 (13:46 +0000)]
oeqa/utils/metadata: Allow to function without the git module
The python git module may or may not be enabled, allow this code to
function without it, falling back to the same method as metadata_scm.bbclass
uses. This will be cleaned up in the next round of feature development.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 29 Oct 2018 13:49:24 +0000 (13:49 +0000)]
testsdk: Improvements to the json logging
Tweak the preceeding commit to:
* Add STARTTIME to the identifier to make it unique
* Add MACHINE to the identifier
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yeoh Ee Peng [Tue, 23 Oct 2018 05:57:22 +0000 (13:57 +0800)]
testsdk.bbclass: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA sdk and sdkext need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.
By default, json testresult file will be written to "oeqa"
directory under the "WORKDIR" directory.
To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 29 Oct 2018 13:49:14 +0000 (13:49 +0000)]
testimage: Improvements to the json logging
Tweak the preceeding commit to:
* Add STARTTIME to the identifier to make it unique
* Log DISTRO
* Use LOG_DIR
* Store the layer config in a more natural json format
* Drop '_' function prefixes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yeoh Ee Peng [Tue, 23 Oct 2018 05:57:21 +0000 (13:57 +0800)]
testimage.bbclass: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA testimage need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.
By default, json testresult file will be written to "oeqa"
directory under the "WORKDIR" directory.
To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yeoh Ee Peng [Tue, 23 Oct 2018 05:57:20 +0000 (13:57 +0800)]
oeqa/selftest/context: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA selftest need to output testresult into json files, where
these json testresult files will be stored into git repository
by the future test-case-management tools.
By default, json testresult file will be written to "oeqa"
directory under the oe-selftest log directory.
To configure multiple instances of bitbake to write json testresult
to a single testresult file at custom directory, user will define
the variable "OEQA_JSON_RESULT_DIR" with the custom directory for
json testresult.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yeoh Ee Peng [Tue, 23 Oct 2018 05:57:19 +0000 (13:57 +0800)]
oeqa/core/runner: write testresult to json files
As part of the solution to replace Testopia to store testresult,
OEQA need to output testresult into single json file, where json
testresult file will be stored in git repository by the future
test-case-management tools.
The json testresult file will store more than one set of results,
where each set of results was uniquely identified by the result_id.
The result_id would be like "runtime-qemux86-core-image-sato", where
it was a runtime test with target machine equal to qemux86 and running
on core-image-sato image. The json testresult file will only store
the latest test content for a given result_id. The json testresult
file contains the configuration (eg. COMMIT, BRANCH, MACHINE, IMAGE),
result (eg. PASSED, FAILED, ERROR), test log, and result_id.
Based on the destination json testresult file directory provided,
it could have multiple instances of bitbake trying to write json
testresult to a single testresult file, using locking a lockfile
alongside the results file directory to prevent races.
Also the library class inside this patch will be reused by the future
test-case-management tools to write json testresult for manual test
case executed.
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Victor Kamensky [Fri, 26 Oct 2018 02:59:55 +0000 (19:59 -0700)]
systemtap: move systemtap-exporter into separate package
stap-exporter runs a set of systemtap scripts and relays
their procfs outputs to remote HTTP clients on demand.
systemtap-exporter is not supposed to run in untrusted
environment. It starts HTTP server on some port. It does
not look safe enough to be included by default along with
the rest of systemtap.
Move systemtap-exporter, its systemd unit, configuration
files and examples scripts into separate package. So if one
needs it and understand its implication, he/she can include
it explicitely.
Signed-off-by: Victor Kamensky <kamensky@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Thu, 25 Oct 2018 06:18:53 +0000 (14:18 +0800)]
package_manager.py: correct the deploydir when packagefeed-stability inherited
After create_packages_dir added in below commit: 85e72e1 package_manager: Filter to only rpms we depend upon
When add below line into conf/local.conf
INHERIT += "packagefeed-stability"
There comes below error when do_rootfs
Exception: FileExistsError: [Errno 17] File exists: '/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm' -> '/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm'
In create_packages_dir function, there is a logic
as bb.utils.remove(subrepo_dir, recurse=True) to
clean subrepo_dir which is actually as example is
/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm.
But currently when inherit packagefeed-stability class,
the deploydir should be /$Prj/tmp/deploy/rpm-prediff,
not the default /$Prj/tmp/deploy/rpm.
If use /$Prj/tmp/deploy/rpm, then result in the
logic as below:
os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm")
Update to the actual deploydir to guarantee the logic
as below:
os.link("/$Prj/tmp/deploy/rpm-prediff/i586/initscripts-1.0-r155.i586.rpm", "/$Prj/tmp/work/qemux86-poky-linux/core-image-minimal/1.0-r0/oe-rootfs-repo/rpm/i586/initscripts-1.0-r155.i586.rpm")
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Mon, 22 Oct 2018 07:03:43 +0000 (15:03 +0800)]
systemd: add back alternatives for init utitilies
Add back alternatives for init utilities to avoid regression.
These alternatives were removed when upgradeing systemd to 239.
They were removed out of the logic that init utitilies should be
bound to init manager. However, it turned out that two use cases
were not covered.
1) initramfs using commands like 'reboot' from busybox.
2) Users use customized busybox defconfig which enables init utilities.
The first use case caused a regression bug in yocto.
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12914
Patches were sent to fix the reboot problem.
But this is not enough. As we may have the second use case. In such
situation, users will find themselves having regression error when
using 'busybox + systemd' (and busybox is installed after systemd,
overriding the systemd symlinks).
So in order to avoid regression, add back these alternatives.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mark Hatle [Thu, 25 Oct 2018 09:58:26 +0000 (05:58 -0400)]
weston: Split out machine specific configuration
Weston needs to be configured to load the fbdev driver when run on a QEMU system.
Other MACHINEs may want to also provider their own configuration as well..
Adding a new RRECOMMEND configuration package will allow this, but avoid
installing empty packages/files in the majority case where it is not needed.
Add maintainer entry as well.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kernel-devsrc: add selinux include files needed by scripts/selinux build
If CONFIG_SECURITY_SELINUX=y is enabled in kernel configuration, then
'make scripts' command in /usr/src/kernel fails to build
utilities under scripts/selinux that would be pulled in by this config:
HOSTCC scripts/selinux/genheaders/genheaders
scripts/selinux/genheaders/genheaders.c:19:10: fatal error: classmap.h: No such file or directory
#include "classmap.h"
To address this issue add security/selinux/include files into
kernel-devsrc.
Signed-off-by: Victor Kamensky <kamensky@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Some distros want to install the kernel in a custom location other
than /boot and have it properly packaged, so it's useful to use a
weaker assignment.
Signed-off-by: Ioan-Adrian Ratiu <adrian.ratiu@ni.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kevin Hao [Mon, 22 Oct 2018 11:58:40 +0000 (19:58 +0800)]
udev-extraconf: Skip the entry in /etc/fstab when using the systemd-mount
When using systemd, the systemd-fstab-generator would parse the
/etc/fstab and create the corresponding unit dynamically. So we don't
need to handle the ADD action for the partitions in /etc/fstab.
Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kevin Hao [Mon, 22 Oct 2018 11:58:39 +0000 (19:58 +0800)]
udev-extraconf: Fix the recursively dependency for the systemd-mount
The commit 4ca9402c3720 ("udev-extraconf: Add systemd-mount to
udev-extraconf/mount.sh") uses the systemd-mount to mount the new added
disk partitions if systemd is used. But it forgot to move the codes
which tries to mount the partition by using the configuration in
/etc/fstab to the non-systemd function. And it will cause the
systemd-mount try to mount the partition synchronously and trigger a
recursively dependency like the following:
dev-sda1.device -> run-media-sda1.mount -> dev-sda1.device
Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kevin Hao [Mon, 22 Oct 2018 11:58:38 +0000 (19:58 +0800)]
udev-extraconf: Use the canonical file name of systemd
The new version of systemd has changed the symbolic link between
/sbin/init and /lib/systemd/systemd to relative. So the output of
the command 'readlink /sbin/init' become:
../lib/systemd/systemd
Then it causes the following check of "/lib/systemd/systemd" to return
false. Fix this issue by using the canonical file name of the systemd.
Signed-off-by: Kevin Hao <kexin.hao@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 22 Oct 2018 22:43:32 +0000 (23:43 +0100)]
npm: change install directory to upstream default
The node binary searches for packages in a number of locations, the last
of which is $PREFIX/lib/node (here: /usr/lib/node) from the list of
GLOBAL_FOLDERS [1]. So change the installation directory for all
packages depending on npm.bbclass to that location. This removes the
need to define the NODE_PATH variable to the non-standard
/usr/lib/node_modules value.
While the Tips for Package Managers [2] discusses installing packages to
/usr/lib/node_modules/<name>/<version>, this has several drawbacks:
* it does not work for the REPL as mentioned in the documentation
* it also does not work for any code _not_ installed as a global
package under /usr/lib/node_modules (e.g. /usr/share/foo.js will not
find any packages below /usr/lib)
* using the non-default location and then having to set NODE_PATH
barely saves any time: there are only two file-system lookups (to the
legacy $HOME/.node_modules and $HOME/.node_libraries) directories
before the library would be found
And the suggestion was made in the context of deduping the node_modules
tree by installing all packages in a flat hierarchy and using symlinks
to the correct version of each dependency. This is not what OpenEmbedded
does, so none of those benefits (deduping, cleaner packages) are being
had by shifting the installation directory to /usr/lib/node_modules.
The choice of a "proper" installation path is not helped by npm
installing to /usr/lib/node_modules if asked to install globally. Still,
using the location expected by nodejs (/usr/lib/node) seems the right
choice.
Randy MacLeod [Mon, 22 Oct 2018 04:21:16 +0000 (00:21 -0400)]
valgrind: Make local functions static to avoid assembler error
Avoid mips32 x-compiler warnings such as:
| ../../../valgrind-3.14.0/helgrind/tests/annotate_hbefore.c:360:6: warning: no previous prototype for 'do_signal' [-Wmissing-prototypes]
| void do_signal ( UWord* w )
| ^~~~~~~~~
by making functions and global variables that are file scope be static
and more importantly also avoid an assembler error:
/tmp/cce22iiw.s: Assembler messages:
/tmp/cce22iiw.s:446: Error: symbol `exit_0' is already defined
/tmp/cce22iiw.s:448: Error: symbol `exit' is already defined
/tmp/cce22iiw.s:915: Error: symbol `exit_0' is already defined
/tmp/cce22iiw.s:917: Error: symbol `exit' is already defined
Signed-off-by: Randy MacLeod <Randy.MacLeod@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Thu, 18 Oct 2018 08:18:05 +0000 (04:18 -0400)]
udev-hwdb: fix postinstall scripts failures when multilib enabled
When multilib is enabled and both udev-hwdb and ${MLPREFIX}udev-hwdb are
installed to image, it fails to run one of their postinstall scripts
that they both call ${base_bindir}/udevadm with same user mode qemu.
Duplicate udevadm and add postinst-intercept update_udev_hwdb to fix the
failures.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oe-selftest: devtool: Support meta being a symbolic link
oe-selftest's devtool tests have been broken since commit 2457cd57
(oe-selftest: devtool: avoid parallel races by using temporary copy of
core) if meta is a symbolic link.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mark Hatle [Wed, 17 Oct 2018 16:43:23 +0000 (12:43 -0400)]
systemd: Remove items that made this machine (qemu) specific
Create a new systemd-conf recipe to contain the specific system/machine
configuration items. This new package is now machine specific.
Without doing this trying to create a single system with multiple BSPs,
one of which was qemu based, would result in the systemd -and- everything that
dependend upon systemd to have their hash changed. The hash changing means
lots of rebuilds, but worse if it's a package based system each different
machine ends with a new PR value and a newly generated package.
Signed-off-by: Mark Hatle <mark.hatle@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Matt Hoosier [Fri, 19 Oct 2018 16:04:55 +0000 (11:04 -0500)]
volatile-binds: use overlayfs if available
Copying files from the read-only root filesystem to the tmpfs
providing the volatile directories can be slow and waste memory.
If the kernel supports the overlay filesystem, use it to mount
a writable tmpfs on top of the read-only directory from the
rootfs and avoid copies.
Ross Burton [Fri, 19 Oct 2018 19:43:18 +0000 (20:43 +0100)]
openssl: do an out-of-tree build
OpenSSL supports out-of-tree builds so we should use them. This makes builds
more reliable, and makes it easier to reduce the size of the ptest package.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ross Burton [Fri, 19 Oct 2018 19:43:17 +0000 (20:43 +0100)]
openssl: fix ptest
Previously the ptest installation was simply a copy of the entire build tree,
which is terribly ugly.
Instead copy just the pieces we need, symlink to /usr as appropriate, and add
missing dependencies. Remove PRIVATE_LIBS as we don't ship copies of the
libraries now.
Also remember to do 'set -x' in run-ptest, so if the tests fail the runner
knows!
[ YOCTO #12965 ]
[ YOCTO #12967 ]
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Olaf Mandel [Fri, 19 Oct 2018 15:22:14 +0000 (15:22 +0000)]
npm: change install directory to upstream default
The node binary searches for packages in a number of locations, the last
of which is $PREFIX/lib/node (here: /usr/lib/node) from the list of
GLOBAL_FOLDERS [1]. Change the installation directory for all packages
depending on npm.bbclass to that location. This removes the need to
define the NODE_PATH variable to the non-standard /usr/lib/node_modules
value.
While the Tips for Package Managers [2] discusses installing packages to
/usr/lib/node_modules/<name>/<version>, this has several drawbacks:
* it does not work for the REPL as mentioned in the documentation
* it also does not work for any code _not_ installed as a global
package under /usr/lib/node_modules (e.g. /usr/share/foo.js will not
find any packages below /usr/lib)
* using the non-default location and then having to set NODE_PATH
barely saves any time: there are only two file-system lookups (to the
legacy $HOME/.node_modules and $HOME/.node_libraries) directories
before the library would be found
And the suggestion was made in the context of deduping the node_modules
tree by installing all packages in a flat hierarchy and using symlinks
to the correct version of each dependency. This is not what OpenEmbedded
does, so none of those benefits (deduping, cleaner packages) are being
had by shifting the installation directory to /usr/lib/node_modules.
Olof Johansson [Fri, 19 Oct 2018 16:15:23 +0000 (18:15 +0200)]
devtool-source.bbclass: Only create each patch branch once
For conditonally applied patches based on SRC_URI overrides, the
devtool-source class would try to create a new branch for each override
assignment as a postfunc to do_patch, but if the same override was used
multiple times, it would try to create the same branch multiple times,
causing errors like
> Exception: bb.process.ExecutionError: Execution of
\ 'git checkout f0f0f0f0f0ff0f0f0f0f0f0f0f0f0f0f0ff0f0f0 -b devtool-override-foo'
\ failed with exit code 128:
> fatal: A branch named 'devtool-override-foo' already exists.
This change makes sure that the devtool-source bbclass will only create
one branch per override.
Signed-off-by: Olof Johansson <olofjn@axis.com> Reviewed-by: Peter Kjellerstedt <pkj@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Fri, 19 Oct 2018 02:37:23 +0000 (10:37 +0800)]
buildtools-tarball: add nativesdk-rpcsvc-proto
Fedora28 repackages rpcgen program to rpcgen
package and the program will no longer be
part of the glibc-common package.
fedora 28:
$ rpm -qf /usr/bin/rpcgen
rpcgen-1.3.1-4.fc28.x86_64
Once build a project on fedora28 host without
installing the extra rpcgen package, there
comes below error:
ERROR: Unable to start bitbake server
ERROR: Last 10 lines of server log for this session (/yocto/builds/upgrade2/bitbake-cookerdaemon.log):
self.cooker = bb.cooker.BBCooker(self.configuration, self.featureset)
File "/yocto/poky/bitbake/lib/bb/cooker.py", line 197, in __init__
self.initConfigurationData()
File "/yocto/poky/bitbake/lib/bb/cooker.py", line 356, in initConfigurationData
self.databuilder.parseBaseConfiguration()
File "/yocto/poky/bitbake/lib/bb/cookerdata.py", line 317, in parseBaseConfiguration
raise bb.BBHandledException
bb.BBHandledException
ERROR: The following required tools (as specified by HOSTTOOLS) appear to be unavailable in PATH, please install them in order to proceed:
rpcgen
So add nativesdk-rpcsvc-proto to provide the
program rpcgen to fix the gap.
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
