Apply patches from systemd_239-7ubuntu10.8 to fix CVE-2019-6454.
CVE-2019-6454 is an issue in which systemd (PID1) can be crashed with a
specially formed D-Bus message.
Richard Purdie [Thu, 21 Feb 2019 16:00:02 +0000 (16:00 +0000)]
kernel: Ensure an initramfs is added if configured
If ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE_NAME}.cpio does not exist,
nor any of the compressed variants, nothing is copied to kernel build's
./usr directory.
The code does not fail, but silently proceeds without a bundled initramfs.
Change to fail and tell the user something is wrong.
Also, if an initramfs is found, contrary to the comments, it does not stop
at the first uncompressed/compressed cpio image found. Instead it keeps
processing all so the last is used. Fix this to behave as per the comments.
[YOCTO #12909]
(Patch by Leon Woestenberg)
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 16 Feb 2019 18:13:00 +0000 (18:13 +0000)]
resulttool: Improvements to allow integration to the autobuilder
This is a combined patch of the various tweaks and improvements I
made to resulttool:
* Avoid subprocess.run() as its a python 3.6 feature and we
have autobuilder workers with 3.5.
* Avoid python keywords as variable names
* Simplify dict accesses using .get()
* Rename resultsutils -> resultutils to match the resultstool ->
resulttool rename
* Formalised the handling of "file_name" to "TESTSERIES" which the code
will now add into the json configuration data if its not present, based
on the directory name.
* When we don't have failed test cases, print something saying so
instead of an empty table
* Tweak the table headers in the report to be more readable (reference
"Test Series" instead if file_id and ID instead of results_id)
* Improve/simplify the max string length handling
* Merge the counts and percentage data into one table in the report
since printing two reports of the same data confuses the user
* Removed the confusing header in the regression report
* Show matches, then regressions, then unmatched runs in the regression
report, also remove chatting unneeded output
* Try harder to "pair" up matching configurations to reduce noise in
the regressions report
* Abstracted the "mapping" table concept used to pairing in the
regression code to general code in resultutils
* Created multiple mappings for results analysis, results storage and
'flattening' results data in a merge
* Simplify the merge command to take a source and a destination,
letting the destination be a directory or a file, removing the need for
an output directory parameter
* Add the 'IMAGE_PKGTYPE' and 'DISTRO' config options to the regression
mappings
* Have the store command place the testresults files in a layout from
the mapping, making commits into the git repo for results storage more
useful for simple comparison purposes
* Set the oe-git-archive tag format appropriately for oeqa results
storage (and simplify the commit messages closer to their defaults)
* Fix oe-git-archive to use the commit/branch data from the results file
* Cleaned up the command option help to match other changes
* Follow the model of git branch/tag processing used by oe-build-perf-report
and use that to read the data using git show to avoid branch change
* Add ptest summary to the report command
* Update the tests to match the above changes
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mazliana [Thu, 14 Feb 2019 05:50:38 +0000 (13:50 +0800)]
scripts/resulttool: enable manual execution and result creation
Integrated “manualexecution” operation to resulttool scripts.
Manual execution script is a helper script to execute all manual
test cases in baseline command, which consists of user guideline
steps and the expected results. The last step will ask user to
provide their input to execute result. The input options are
passed/failed/blocked/skipped status. The result given will be
written in testresults.json including log error from the user
input and configuration if there is any.The output test result
for json file is created by using OEQA library.
The configuration part is manually key-in by the user. The system
allow user to specify how many configuration they want to add and
they need to define the required configuration name and value pair.
In QA perspective, "configuration" means the test environments and
parameters used during QA setup before testing can be carry out.
Example of configurations: image used for boot up, host machine
distro used, poky configurations, etc.
The purpose of adding the configuration is to standardize the
output test result format between automation and manual execution.
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To execute manual test cases, execute the below
$ resulttool manualexecution <manualjsonfile>
By default testresults.json store in <build_dir>/tmp/log/manual/
[YOCTO #12651]
Signed-off-by: Mazliana <mazliana.mohamad@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Yeoh Ee Peng [Thu, 14 Feb 2019 05:50:37 +0000 (13:50 +0800)]
resulttool: enable merge, store, report and regression analysis
OEQA outputs test results into json files and these files were
archived by Autobuilder during QA releases. Example: each oe-selftest
run by Autobuilder for different host distro generate a
testresults.json file.
These scripts were developed as a test result tools to manage
these testresults.json file.
Using the "store" operation, user can store multiple testresults.json
files as well as the pre-configured directories used to hold those files.
Using the "merge" operation, user can merge multiple testresults.json
files to a target file.
Using the "report" operation, user can view the test result summary
for all available testresults.json files inside a ordinary directory
or a git repository.
Using the "regression-file" operation, user can perform regression
analysis on testresults.json files specified. Using the "regression-dir"
and "regression-git" operations, user can perform regression analysis
on directory and git accordingly.
These resulttool operations expect the testresults.json file to use
the json format below.
{
"<testresult_1>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
...
"<testresult_n>": {
"configuration": {
"<config_name_1>": "<config_value_1>",
"<config_name_2>": "<config_value_2>",
...
"<config_name_n>": "<config_value_n>",
},
"result": {
"<testcase_namespace_1>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
"<testcase_namespace_2>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
...
"<testcase_namespace_n>": {
"status": "<PASSED or FAILED or ERROR or SKIPPED>",
"log": "<failure or error logging>"
},
}
},
}
To use these scripts, first source oe environment, then run the
entry point script to look for help.
$ resulttool
To store test result from oeqa automated tests, execute the below
$ resulttool store <source_dir> <git_branch>
To merge multiple testresults.json files, execute the below
$ resulttool merge <base_result_file> <target_result_file>
To report test report, execute the below
$ resulttool report <source_dir>
To perform regression file analysis, execute the below
$ resulttool regression-file <base_result_file> <target_result_file>
To perform regression dir analysis, execute the below
$ resulttool regression-dir <base_result_dir> <target_result_dir>
To perform regression git analysis, execute the below
$ resulttool regression-git <source_dir> <base_branch> <target_branch>
[YOCTO# 13012]
[YOCTO# 12654]
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Anuj Mittal [Wed, 20 Feb 2019 07:12:47 +0000 (15:12 +0800)]
glib-networking: upgrade 2.54.1 -> 2.58.0
* Autotools support has been removed upstream, so migrate recipe to
meson. For changes, see:
https://gitlab.gnome.org/GNOME/glib-networking/blob/glib-2-58/NEWS
Mingli Yu [Wed, 13 Feb 2019 05:43:28 +0000 (13:43 +0800)]
logrotate.py: restore /etc/logrotate.d/wtmp
During the test logrotate.LogrotateTest.test_1_logrotate_setup,
there is below logic:
# mkdir $HOME/logrotate_dir
# sed -i "s#wtmp {#wtmp {\n olddir $HOME/logrotate_dir#" /etc/logrotate.d/wtmp
After all logrotate.LogrotateTest finished, only cleanup
$HOME/logrotate_dir as below, but don't restore
the config file /etc/logrotate.d/wtmp.
[snip]
def tearDownClass(cls):
cls.tc.target.run('rm -rf $HOME/logrotate_dir')
[snip]
That's to say, there is one additional line added
to /etc/logrotate.d/wtmp and will make the logrotate
service start failed when run systemd.SystemdBasicTests.test_systemd_failed
Take an example as below when run test as root:
# cat /etc/logrotate.d/wtmp
# no packages own wtmp -- we'll rotate it here
/var/log/wtmp {
olddir /root/logrotate_dir
missingok
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
# ls /root/logrotate_dir
ls: cannot access '/root/logrotate_dir': No such file or directory
# systemctl start logrotate
Job for logrotate.service failed because the control process exited with error code.
See "systemctl status logrotate.service" and "journalctl -xe" for details.
# systemctl status logrotate
logrotate.service - Rotate log files
Loaded: loaded (/lib/systemd/system/logrotate.service; static; vendor preset>
Active: failed (Result: exit-code) since Wed 2019-02-13 03:35:19 UTC; 7s ago
Docs: man:logrotate(8)
man:logrotate.conf(5)
Process: 540 ExecStart=/usr/sbin/logrotate /etc/logrotate.conf (code=exited, status=1/FAILURE)
Main PID: 540 (code=exited, status=1/FAILURE)
Feb 13 03:35:18 qemumips systemd[1]: Starting Rotate log files...
Feb 13 03:35:19 qemumips logrotate[540]: error: wtmp:9 error verifying olddir path /root/logrotate_dir: No such file or directory
Feb 13 03:35:19 qemumips logrotate[540]: error: found error in file wtmp, skipping
Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Main process exited, code=exited, status=1/FAILURE
Feb 13 03:35:19 qemumips systemd[1]: logrotate.service: Failed with result 'exit-code'.
Feb 13 03:35:19 qemumips systemd[1]: Failed to start Rotate log files.
Add the logic to restore /etc/logrotate.d/wtmp to
make the cleanup complete to fix the above issue.
Signed-off-by: Mingli Yu <Mingli.Yu@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Changqing Li [Tue, 19 Feb 2019 01:10:33 +0000 (09:10 +0800)]
python3: fix INCLDIRSTOMAKE in _sysconfigdata.py
recipe glade do_configure failed after python upgrade
to 3.7.2, it will do runtime check of python headers
with native python3 under recipe-sysroot-native, it will check under
path INCLDIRSTOMAKE, but this value is '/usr/include /usr/include
/usr/include/python3.7m /usr/include/python3.7m', which will cause
below error:
ERROR: This autoconf log indicates errors, it looked at host include and/or
library paths while determining system capabilities.
fix it by replace /usr/include to STAGING_INCDIR
Signed-off-by: Changqing Li <changqing.li@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libc-package.bbclass: Add a progress meter for the package task
The package task for glibc-locale takes a very long time to execute,
especially if using qemu. In that case, a progress meter helps a lot to
show the progress of the task.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
libc-package.bbclass: Only extract wanted files in prep_locale_tree()
prep_locale_tree() predates the usrmerge DISTRO_FEATURE, which meant it
was not prepared for the case when ${base_libdir} == ${libdir}. This
lead to it extracting files and directories where it shouldn't.
Signed-off-by: Peter Kjellerstedt <peter.kjellerstedt@axis.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 17 Feb 2019 22:12:40 +0000 (22:12 +0000)]
scripts/oe-git-archive: Separate out functionality to library function
This turns the core of the script into a library function. Ultimately this
will let us call that code with custom 'keywords' rather than relying
on the data parsed from bitbake metadata which can't be used when archiving
historical results.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Manjukumar Matha [Fri, 15 Feb 2019 00:45:18 +0000 (16:45 -0800)]
kernel-fitimage.bbclass: Fix the dependency issue while generating fitimage_initramfs
When building fitimage_initramfs, the correct depedency is to build
after do_bundle_initramfs. We can run into the following dependency
issue
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing shell function do_assemble_fitimage_initramfs
aarch64-xilinx-linux-objcopy: 'vmlinux': No such file
This happens because initramfs renames vmlinux to vmlinux.bak while
generating vmlinux.initramfs, there is a chance that fitimage_initramfs
can also start during this process and create the above issue.
This patch resolve the dependency issue by running fitimage_initramfs
task after do_bundle_initramfs
Alex Kiernan [Sat, 16 Feb 2019 13:41:55 +0000 (13:41 +0000)]
bmap-tools: update from 3.4 to 3.5
bmap-tools 3.5 was released August 2018 but has no release tarball, so
switch to using the git fetcher. Also pull in the fix for StopIteration
being transformed into RuntimeError for Python 3.7.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 16 Feb 2019 15:00:45 +0000 (15:00 +0000)]
quilt: Merge recipe files into a more coherent form
The style of this recipe is dated, move most of the code into the main
shared include file, making some of the configuration much clearer using
modern overrides to do so.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 16 Feb 2019 14:52:29 +0000 (14:52 +0000)]
native: Enable RDEPENDS handling
Native recipes don't currently honour their RDEPENDS. In the case of
some python scripts this has started causing problems since whilst they're
not needed at build time (DEPENDS), they are needed at runtime.
We put off making this change due to circular dependency issues. I believe
the three such problems in OE-Core are now fixed, as is the dependency loop
identfication code in bitbake so its time to improve this situation.
[YOCTO #10113]
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 16 Feb 2019 14:48:29 +0000 (14:48 +0000)]
quilt-native: Remove RDEPENDS on util-linux-native
This code is actually inactive and inaccurate. If we fix other code to
enable native RDEPENDS handling, this then breaks.
quilt-native is early in the dependency tree and can't depend on
util-linux-native which may need to applu patches. The pieces of
util-linux which quilt needs are long established as part of HOSTTOOLS.
The other RDEPENDS are already part of ASSUME_PROVIDED. util-linux-native
doesn't belong there since it has multiple components and is used as a valid
dependency elsewhere in the codebase.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 16 Feb 2019 14:01:56 +0000 (14:01 +0000)]
lib/oe/utils: Fix hang in multiprocess_launch()
If large results values are returned by the subprocesses, we can hit a deadlock
where the subprocess is trying to write data back to the parent, the pipe is full
and the parent is waiting for the child to exit.
Avoid this by calling the update() method which would trigger reading a result
from the child, avoiding the deadlock. The issue is described in
https://bugs.python.org/issue8426
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 30 Jan 2019 17:42:25 +0000 (17:42 +0000)]
package: Rework debug source file handling
Currently we parallel process the files we install running dwarfsrcfiles over
each one in parallel threads but requiring a lock to write the results to one
file. This is not ideal for performance and means we can't then use per file
data for other purposes such as source code license processing.
Rework the code so that the list of source files is generated per installed
file and is reusable.
The code still generates a null separated debugsources.list file since this
is used by a shell pipeline but it no longer needs locking.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Adam Trhon [Tue, 12 Feb 2019 12:14:58 +0000 (12:14 +0000)]
icecc-env: don't raise error when icecc not installed
When icecc is not installed, the `which icecc` command in icecc-env.sh
returns nonzero. This happens when environment is being sourced. When
the terminal has `set -e`, the whole script fails and terminal is
closed.
Fix this by ignoring errors from the which command.
Signed-off-by: Adam Trhon <adam.trhon@tbs-biometrics.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Pascal Bach [Thu, 14 Feb 2019 16:15:29 +0000 (17:15 +0100)]
nfs-utils: build tools with target compiler
Some tools were built with CC_FOR_BUILD which points to the target
compiler. The current patch avoided issues by deleting some of the
binaries during install.
This patch replaces the CC_FOR_BUILD with CC so the tools are built with
the target compiler. This means the binaries no longer need to be
deleted.
I stumbled upon this by trying to globally add "--ffile-prefix-map", which
is not supported by my host GCC, to get rid of some "buildpaths" QA Warnings.
Cc: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Pascal Bach <pascal.bach@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alex Kiernan [Tue, 15 Jan 2019 06:55:07 +0000 (06:55 +0000)]
perf: Package libperf-jvmti.so as part of ${PN}
perf's jvmti support builds libperf-jvmti.so that needs to be packaged
as part of the main package, not a developer package. Given there's no
files in the developer package, override FILES_SOLIBSDEV so it's not
gobbled by FILES_${PN}-dev, and then add just that file to FILES_${PN}
so that if in the future there other files which would be added to the
dev package we'll fail then.
Signed-off-by: Alex Kiernan <alex.kiernan@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 15 Feb 2019 15:58:49 +0000 (15:58 +0000)]
target-sdk-provides-dummy: Extend to -dev and -src packages
This avoids errors when running populate_sdk under opkg:
* Problem 1/1:
* - package busybox-dev-1.30.1-r0.core2-64 requires busybox = 1.30.1-r0, but none of the providers can be installed
*
* Solution 1:
* - allow deinstallation of target-sdk-provides-dummy-1.0-r0.sdk-provides-dummy-target
* Solution 2:
* - do not ask to install a package providing busybox-dev
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Fri, 15 Feb 2019 02:06:38 +0000 (21:06 -0500)]
openssl: update patch to fix buildpaths qa issue for -fmacro-prefix-map
Gcc option '-fmacro-prefix-map' is added to DEBUG_PREFIX_MAP. It has a
patch to deal option '-fdebug-prefix-map' already. Update the patch
0001-buildinfo-strip-sysroot-and-debug-prefix-map-from-co.patch to fix
buildpaths qa issue for '-fmacro-prefix-map' too.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Fri, 15 Feb 2019 02:06:36 +0000 (21:06 -0500)]
go-cross-canadian: set default value of DEBUG_PREFIX_MAP
It uses gcc on build machine during go-cross-canadian bootstrap, but the
gcc version may be old and not support option '-fmacro-prefix-map' which
is one of default values of DEBUG_PREFIX_MAP.
So set default value of DEBUG_PREFIX_MAP for go-cross-canadian.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Fri, 15 Feb 2019 02:06:35 +0000 (21:06 -0500)]
bitbake.conf: update DEBUG_PREFIX_MAP with -fmacro-prefix-map
Add option '-fmacro-prefix-map' to DEBUG_PREFIX_MAP. It could resolve
buildpaths qa warnings which caused by macros '__FILE__' and
'__BASE_FILE__'.
Though option '-ffile-prefix-map' could cover '-fdebug-prefix-map' and
'-fdebug-prefix-map' both, it is not supported by 'as' and causes
buildpaths qa warnings of glibc:
| QA Issue: File
| /work/i586-poky-linux/glibc/2.29-r0/packages-split/glibc-dev/usr/lib/crti.o
| in package contained reference to tmpdir
So only add '-fmacro-prefix-map' to DEBUG_PREFIX_MAP rather than replace
'-fdebug-prefix-map' with '-ffile-prefix-map'.
Signed-off-by: Kai Kang <kai.kang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Yang [Fri, 15 Feb 2019 08:14:51 +0000 (16:14 +0800)]
runqemu: Let qemuparams override default settings
Fixed:
In meta/conf/machine/include/qemuboot-x86.inc:
QB_CPU_x86-64 = "-cpu core2duo"
$ runqemu qemux86-64 qemuparams="-cpu coreduo"
Check /proc/cpuinfo, it should use coreduo rather than core2duo since user
specifies it, but it doesn't, append qemuparams to the last can fix the
problem.
[YOCTO #11773]
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
