Bruce Ashfield [Mon, 17 May 2021 22:16:52 +0000 (18:16 -0400)]
kernel-yocto: provide debug / summary information for metadata
It was mentioned that when developing a BSP, the information about what
definition was used, or what fragments have been applied is not obvious
and requires looking at the code.
With this change, we can trigger a full summary of the meta data gathering
phase when KCONF_AUDIT_LEVEL > 0.
Sample output follows:
NOTE: do_kernel_metadata: for summary/debug, set KCONF_AUDIT_LEVEL > 0
NOTE: kernel meta data summary for qemux86-64 (standard):
NOTE:
======================================================================
NOTE: BSP entry point / definition:
/build/tmp/work/qemux86_64-poky-linux/linux-yocto/5.10.34+gitAUTOINC+bca3bfbc74_85c17ad073-r0/kernel-meta/bsp/common-pc-64/common-pc-64-standard.scc
NOTE: Fragments from SRC_URI:
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/xt-checksum.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/ebtables.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/vswitch.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/lxc.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/docker.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/cgroup-hugetlb.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/xen.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/kubernetes.scc
NOTE: KERNEL_FEATURES: features/nfsd/nfsd-enable.scc
features/debug/printk.scc features/kernel-sample/kernel-sample.scc
features/netfilter/netfilter.scc cfg/virtio.scc
features/drm-bochs/drm-bochs.scc cfg/sound.scc cfg/paravirt_kvm.scc
features/scsi/scsi-debug.scc features/gpio/mockup.scc
features/aufs/aufs-enable.scc cfg/fs/flash_fs.scc cfg/virtio.scc
NOTE: Final scc/cfg list:
/build/tmp/work/qemux86_64-poky-linux/linux-yocto/5.10.34+gitAUTOINC+bca3bfbc74_85c17ad073-r0/kernel-meta/bsp/common-pc-64/common-pc-64-standard.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/xt-checksum.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/ebtables.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/vswitch.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/lxc.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/docker.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/cgroup-hugetlb.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/xen.scc
/poky/meta-virtualization/recipes-kernel/linux/linux-yocto/kubernetes.scc
features/nfsd/nfsd-enable.scc features/debug/printk.scc
features/kernel-sample/kernel-sample.scc
features/netfilter/netfilter.scc cfg/virtio.scc
features/drm-bochs/drm-bochs.scc cfg/sound.scc cfg/paravirt_kvm.scc
features/scsi/scsi-debug.scc features/gpio/mockup.scc
features/aufs/aufs-enable.scc cfg/fs/flash_fs.scc cfg/virtio.scc
Signed-off-by: Bruce Ashfield <bruce.ashfield@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
This is not enabled or tested by default, and has never been
ported to python 3 upstream[1], which means it doesn't work at all
with plain poky. If you need it, please put it in a separate layer
and/or modernize to work with py3.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Jacob Kroon <jacob.kroon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mike Crowe [Fri, 14 May 2021 10:38:00 +0000 (11:38 +0100)]
libnotify: Make gtk+3 dependency optional
libnotify only requires gtk+3 for its tests. Let's disable them by
default and only enable them if "tests" is in PACKAGECONFIG. If gtk+3 is
not available then we need to declare the dependency on gdk-pixbuf
explicitly.
It looks like the tests genuinely do need some sort of desktop
environment to run, so let's maintain the ANY_OF_DISTRO_FEATURES check
added back in 3edf08b38b0af93cef0933b061349264dc86d54c.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Cc: Khem Raj <raj.khem@gmail.com> Cc: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 14 May 2021 17:06:56 +0000 (18:06 +0100)]
sstate: Handle manifest 'corruption' issue
Under certain build patterns, warnings about missing manifests can appear. These
are real issues where the manifest was removed and shouldn't have been.
Martin Jansa was able to find a reproducer of:
MACHINE=qemux86 bitbake zlib-native
echo 'PR = "r1"' >> meta/recipes-core/zlib/zlib_1.2.11.bb
MACHINE=qemux86-64 bitbake zlib-native
MACHINE=qemux86 bitbake zlib-native
<the zlib-native manifest is now removed along with the sysroot-components contents>
The code maintains a per machine list of stamps but a per PACAGE_ARCH list of
stamp/manifest/workdir mappings. The latter is only appended to for speed with
the assumption that once stamps are gone, the code wouldn't trigger.
The code only ever appends to the mapping list (for speed/efficency under lock)
meaning that multiple entries can result where the stamp/workdir differs due to
version changes but the manifest remains the same.
By switching MACHINE part way through the build, the older stamp is referenced
and the manifest is incorrectly removed as it matches an now obsolete entry in
the mapping file.
There are two possible fixes, one is to rewrite the mapping file every time
which means adding regexs, iterating and generally complicating that code. The
second option is to only use the last mapping entry in the file for a given
manifest and ignore any earlier ones. This patch implments the latter.
Also drop the stale entries if we are rewriting it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mike Crowe [Thu, 13 May 2021 14:25:03 +0000 (15:25 +0100)]
npm.bbclass: Allow nodedir to be overridden by NPM_NODEDIR
Node modules may need to be built against multiple Node
versions. Setting nodedir in the NPM configuration stops older ways of
doing this, such as setting npm_config_target and npm_config_disturl,
from working.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop 0001-Add-output-of-tests-result.patch
(difficult to rebase). I have verified that ptests
still pass, and print PASS for every test. If they
start failing we can revisit what kind of output would
be beneficial.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 11 May 2021 10:47:17 +0000 (11:47 +0100)]
qemu-x86: Add commandline options to improve boot
To try and help with the inermittent rcu stalls and boot issues with qemu
add some kernel commandline options commonly used with qemu instances which
were suggested may help.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 11 May 2021 09:50:10 +0000 (10:50 +0100)]
qemuboot-x86: Switch to IvyBridge and q35 instead of pc
Move from 1996 to 2009 by swapping machine 'pc' for 'q35'.
Also move to a CPU which is SMP capable and doesn't have tsc bugs. IvyBridge
matches what we're using on the autobuilder.
The intent here is to try and improve on some of the intermittent autobuilder
issues we're seeing. I'm told that nobody else runs with config this old
and it could well be contributing to our issues. Having reliable testing
is key to the project and justifies updating this IMO.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sat, 8 May 2021 09:42:22 +0000 (10:42 +0100)]
qemu: Set SMP to 4 cpus for arm/x86 only
Only qemux86* and qemuarm* support SMP with our current configurations so
rework qemu SMP enabling to account for that and only use it on the architectures
where it works.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Robert Joslyn [Wed, 12 May 2021 04:17:17 +0000 (21:17 -0700)]
btrfs-tools: Update to 5.12
Add PACKAGECONFIG for zoned block devices. Leave disabled by default, as
this requires a recent kernel.
License-Update: libbtrfsutil was relicensed to LGPLv2.1+: https://git.kernel.org/pub/scm/linux/kernel/git/kdave/btrfs-progs.git/commit/?id=9e30f779e3cbf12a88ebe309ee371851fdfd3aa0
Signed-off-by: Robert Joslyn <robert.joslyn@redrectangle.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Wed, 12 May 2021 16:45:03 +0000 (18:45 +0200)]
binutils: backport DWARF-5 support for gold
* DWARF-5 is now used by default with gcc-11, causing
11.1.0/ld: internal error in format_file_lineno, at ../../gold/dwarf_reader.cc:2278
collect2: error: ld returned 1 exit status
in various projects (runc-opencontainers, libhybris, collada-dom)
* https://gcc.gnu.org/gcc-11/changes.html
For targets that produce DWARF debugging information GCC now defaults to DWARF version 5
(with the exception of VxWorks and Darwin/Mac OS X which default to version 2 and
AIX which defaults to version 4). This can produce up to 25% more compact debug
information compared to earlier versions.
To take full advantage of DWARF version 5 GCC needs to be build against binutils version 2.35.2 or higher.
When GCC is build against earlier versions of binutils GCC will still emit DWARF version 5 for
most debuginfo data, but will generate version 4 debug line tables (even when explicitly given -gdwarf-5).
The following debug information consumers can process DWARF version 5:
GDB 8.0, or higher
valgrind 3.17.0
elfutils 0.172, or higher (for use with systemtap, dwarves/pahole, perf and libabigail)
dwz 0.14
Programs embedding libbacktrace are urged to upgrade to the version shipping with GCC 11.
To make GCC 11 generate an older DWARF version use -g together with -gdwarf-2, -gdwarf-3 or -gdwarf-4.
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
An XML parser (either libxml2, or expat) is only needed by libarchive to
read/write XAR archives. However, these also need OpenSSL enabled which
by default it is not, so XAR files were disabled and libxml2 was a build
dependency for no reason.
As XAR archives are mainly used on macOS, we can remove libxml2.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 12 May 2021 21:28:45 +0000 (22:28 +0100)]
ovmf: Fix other reproducibility issues
When building in longer paths, the ovmf build changes in many ways. This adds a
patch addressing various causes of problems. Full details are in the patch header.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 12 May 2021 21:28:23 +0000 (22:28 +0100)]
ovmf: Disable lto to aid reproducibility
lto tends to break reproducibility and makes ovmf near impossible to debug
reproducibility issues in. Disable it and supress the warnings that then
generates from Werror.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop mips64_cpu_detection.patch as upstream
has changed the code in a way that's difficult
to rebase. I have confirmed that builds on qemumip64 still
work, and the patch does not say clearly what was the
way to reproduce the failure it's aiming to address.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 11 May 2021 13:13:08 +0000 (14:13 +0100)]
tiff: Exclude CVE-2015-7313 from cve-check
Some fix upstream addresses the issue, it isn't clear which change this was. Our
current version doesn't have issues with the test image though so we can exclude.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Drop configure() tweaks, none of them are needed anymore, and particularly
the ncurses tweak was changing the build in a way that is not clear
(and no one remembers why it was there in the first place).
Adjust LDFLAGS as somehow -pthread isn't there --> link failures.
Signed-off-by: Alexander Kanavin <alex.kanavin@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Sun, 9 May 2021 15:53:07 +0000 (16:53 +0100)]
ovmf: Improve reproducibility by enabling prefix mapping
We want to pass ${DEBUG_PREFIX_MAP} to gcc commands and also pass in
--debug-prefix-map to nasm (we carry a patch to nasm for this). The
tools definitions are built by ovmf-native so we need to pass this in
at target build time when we know the right values.
By using determininistc file paths in the ovmf build, it removes the
opportunitity for gcc/ld to change the output binaries due to path
lengths overflowing section sizes and causing small changes in the
binary output.
This also means that if builds have reproducibility issues in future, it
becomes much easier to compare intermediate build artefacts.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Mon, 10 May 2021 15:30:56 +0000 (16:30 +0100)]
jquery: Exclude CVE-2007-2379 from cve-check
The CVE is non-specific and depends on the users of jquery, doesn't
make sense to have this flagged against jquery as there is nothing we can
do about it.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
