python2-native executable is not available in sysroot anymore, which
causes compilation of some nodejs modules to fail. Switch to python3 as a
default python version.
Signed-off-by: Vyacheslav Yurkov <uvv.mail@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 27 Jan 2021 16:13:29 +0000 (16:13 +0000)]
meta: Clean up various class-native* RDEPENDS overrides
With PACKAGES functioning more correctly for native recipes combined
with classextend improvements over the years, there are various overrides
of RDEPENDS which look unecessary now, clean them up.
There some some minor changes in dependencies, specifically:
Richard Purdie [Wed, 27 Jan 2021 14:48:21 +0000 (14:48 +0000)]
native: Stop clearing PACKAGES
Native recipes have been special and they don't have packages generated
from them. The RDEPENDS/RPROVIDES and other runtime package specific
variables can contain important data about dependencies recipes need
though and currently it is required to write this information explicitly
in the native case.
We now delete the packaging tasks for native recipes which removes the
need to clear PACKAGES. The next step to improve the metadata is to
stop clearing it and ensure any entries in these variables are remapped
appropriately. The R* variables were already being processed by the class
extension code but the implementation was suboptimal.
This patch stops clearing PACKAGES and PACKAGES_DYNAMIC and fixes the places
where that caused issues in OE-Core, for example PACKAGES additions in anonymous
python without the "-native" suffix and a case where the included classes
caused a self reference in DEPENDS which would once have been removed by
the previous code.
The implementation uses datastore/parser parameters to ensure that the
variable overrides are not overwritten when calling setVar which is appropriate
for a function as close to the core as this one is.
Some now unneeded code in python3-setuptools is dropped, there are further
changes like this which can follow.
This change was verified with OE-Core by comparing task-depends.dot generated
by "bitbake world -g" before and after the change, the files were identical.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 27 Jan 2021 21:48:54 +0000 (21:48 +0000)]
package_rpm: Clean up unset runtime package variable handling
If the R* runtime package variables are unset it could cause trackbacks. There
were some fallbacks already, clean this up to handle consistently.
The code was expecting strings but setting defaults of empty lists
which silently were converted to strings by the "or" statements which
was a nightmare to understand or alter.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Peter Bergin [Fri, 29 Jan 2021 09:50:54 +0000 (10:50 +0100)]
buildhistory.bbclass: avoid exception for empty BUILDHISTORY_FEATURES variable
An exception is fired when a BuildStarted event is sent to buildhistory bbclass
and the variable BUILDHISTORY_FEATURES is not set.
ERROR: Execution of event handler 'buildhistory_eventhandler' failed
Traceback (most recent call last):
File "<...>/meta/classes/buildhistory.bbclass", line 862, in buildhistory_eventhandler(e=<bb.event.BuildStarted object at 0x7f94c3810250>):
python buildhistory_eventhandler() {
> if e.data.getVar('BUILDHISTORY_FEATURES').strip():
reset = e.data.getVar("BUILDHISTORY_RESET")
AttributeError: 'NoneType' object has no attribute 'strip'
This can happen in a multiconfig build where the default configuration use the
buildhistory class but not the configuration in mc. It should be a rare case that
this happens and it was found in a missconfigured build.
Signed-off-by: Peter Bergin <peter@berginkonsult.se> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jose Quaresma [Sun, 24 Jan 2021 18:26:45 +0000 (18:26 +0000)]
spirv-tools: cleanup
- spirv-headers is provived in a separete receipe, so add it as dependencie
- add a new package spirv-tools-lesspipe that needs bash
- remove unneeded external sources
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jose Quaresma [Sun, 24 Jan 2021 18:07:58 +0000 (18:07 +0000)]
shaderc: fix the build with glslang 11.1.0
Drop patches:
* 0001-Fix-the-link-order-of-libglslang-and-libHLSL.patch
is rejected upstream and is not need when glslang as shared libs.
Update pacthes:
* 0001-cmake-de-vendor-libs-and-disable-git-versioning.patch
renamed and refreshed from 0003-cmake-de-vendor-libs-and-disable-git-versioning.patch.
Signed-off-by: Jose Quaresma <quaresma.jose@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 27 Jan 2021 17:15:17 +0000 (17:15 +0000)]
staging: Clean up files installed into the sysroot
There are a variety of files being installed into $datadir which we
don't need. Pick the top "offenders" which amount of thousands of files
and simply don't install them. These include things like test data,
terminfo data, locale data for native tools and so on. This saves
copying these files into native and target sysroots and should improve
performance (smaller sstate, fewer files to copy around).
With this and the python recipe change, alsa-tools went from:
Richard Purdie [Wed, 27 Jan 2021 17:12:46 +0000 (17:12 +0000)]
python3: Avoid installing test data into recipe-sysroot
There are several thousand files in the test directory which we don't need.
Adding these for the native and target sysroots is a crazy amount of files
to be throwing around needlessly. Delete the files from the sysroot side
of things to tidy up the sysroots and improve performance.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 27 Jan 2021 12:13:28 +0000 (12:13 +0000)]
ncurses: Don't put terminfo into the sysroot
This recudes the file count from ~2850 to ~100 which is a huge win
for reducing build directory clutter, its unlikely anything uses the
terminfo data or man pages in the sysroot. This is especially helpful
as we usually end up with two copies of these sets of files.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Teoh Jay Shen [Wed, 27 Jan 2021 01:36:13 +0000 (09:36 +0800)]
oeqa/ethernet_ip_connman : add test for network connections
This test mimic the ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP test case from oeqa/manual/bsp-hw.json.
The ethernet_static_ip_set_in_connman and ethernet_get_IP_in_connman_via_DHCP manual test case should be remove from oeqa/manual/bsp-hw.json if this patch get merged.
Signed-off-by: Teoh Jay Shen <jay.shen.teoh@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
lib/oe/package_manager: Do not pass stderr to package manager as an argument
The cmd redirected stderr to stdout that was then assigned to variable
with pkgs to install. Then this variable was passed to package manager
that then tried to install it and generated confusing warnings.
For example this variable could contain:
| ['(en_US.UTF-8)', 'LC_ALL:', 'bash:', 'cannot', 'change', 'locale', 'setlocale:', 'warning:']
Dorinda [Tue, 26 Jan 2021 01:33:59 +0000 (02:33 +0100)]
sanity.bbclass: sanity check for if bitbake is present in PATH
If a user executes the environment script instead of sourcing it,
there's an error about an empty element in PATH. This is because
bitbake isn't present in environment variable PATH. Hence, this
patch adds a sanity check to verify if bitbake is present in
PATH and if bitbake isn't present issue a warning message.
[YOCTO #13822]
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Dorinda [Tue, 26 Jan 2021 00:53:47 +0000 (01:53 +0100)]
sanity: Verify that user isn't building in PSEUDO_IGNORE_PATHS
If a user builds in a path in PSEUDO_IGNORE_PATHS, random failures
are generated. Hence this patch adds a sanity check in sanity.bbclass
to ensure that a user isn't building in PSEUDO_IGNORE_PATHS.
[YOCTO #14179]
Signed-off-by: Dorinda Bassey <dorindabassey@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Alan Perry [Mon, 25 Jan 2021 07:21:19 +0000 (07:21 +0000)]
iproute2: Add subpackage for rdma command
The rdma command is part of iproute2 and is used to query or set the RDMA
configuration where applicable. This patch adds options to build it
and include it.
Signed-off-by: Alan Perry <alanp@snowmoose.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
selftest/devtool: Add modify_localfiles_only test checking symlink path
If recipe uses only file:// fetcher devtool unpacks sources under
oe-local-files/ and adds symlink in source root directory. We need to
verify if the symlink in subdirectory has correct path.
See [YOCTO #13738] for details.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Lee Chee Yang [Fri, 22 Jan 2021 10:07:19 +0000 (18:07 +0800)]
cve-check: replace Looseversion with custom version class
The way distutils.version.LooseVersion compare version are tricky, it treat
all these ( "1.0-beta2", "1.0-rc1", "1.0A", "1.0p2" and "1.0pre1") as greater
version than "1.0". This might be right for "1.0A" and "1.0p1" but not for
the rest, also these version could be confusing, the "p" in "1.0p1" can be
"pre" or "patched" version or even other meaning.
Replace Looseversion with custom class, it uses regex to capture common
version format like "1.1.1" or tag format using date like "2020-12-12" as
release section, check for following known string/tags ( beta, rc, pre, dev,
alpha, preview) as pre-release section, any other trailing characters
are difficult to understand/define so ignore them. Compare release
section and pre-release section saperately.
included selftest for the version class.
[YOCTO#14127]
Signed-off-by: Lee Chee Yang <chee.yang.lee@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Kai Kang [Thu, 21 Jan 2021 07:29:22 +0000 (15:29 +0800)]
adwaita-icon-theme: add version 3.34.3 back
Newer versions of adwaita-icon-theme can't work well with librsvg 2.40.
Boot up the core-image-sato image, and start "Media Player", most of the
icons are not showed properly.
The source code of librsvg has been adapted to rust, so it is impossible
to update librsvg in oe core for now. Then add adwaita-icon-theme 3.34.3
back and set as preferred version.
Duplicate one patch as well to fix patch fuzz issue.
Ross Burton [Thu, 21 Jan 2021 16:09:22 +0000 (16:09 +0000)]
base: use URI instead of decodeurl when detecting unpack dependencies
decodeurl() has limitations, primarily that it doesn't handle query
parameters at all. If a SRC_URI looks like this:
http://example.com/download.tar.gz?something
Then the returned path attribute is download.tar.gz?something. This means
the filename extension detection fails and required tools are not added
to the dependencies.
Signed-off-by: Ross Burton <ross.burton@arm.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
externalsrc: Fix parsing error with devtool non-git sources
If srcdir is under poky directory (e.g. devtool poky/build/workspace/sources)
and is not a git repository then ${@srctree_hash_files(d)} will run "git
rev-parse --git-dir" and detect poky directory as git-dir and run "'git', 'add',
'-A', '.'], cwd=s_dir" trying to add srcdir but build dir is in .gitignore and
latest git will fail with "The following paths are ignored by one of your
.gitignore files: build" which will end with "ExpansionError during parsing".
In this commit I added a check if git_dir is the same as git-dir from
TOPDIR (which will detect poky directory) and if yes, then treat srcdir
as non-git sources.
Signed-off-by: Tomasz Dziendzielski <tomasz.dziendzielski@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
- libgloss:
- (22) New tcl license for csky-* targets
- (23) New license for csky-* targets, states itself as FreeBSD license
but text matches GPL (648365e0)
- (24) New BSD-2 license for risc-v targets
newlib:
- New license for csky-* targets, states itself as FreeBSD license
but text matches GPL (648365e0)
- (53) arm targets FreeBSD
- (54) arm targets BSD-2-Clause-FreeBSD
Signed-off-by: Alejandro Enedino Hernandez Samaniego <alejandro@enedino.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mingli Yu [Thu, 21 Jan 2021 07:44:49 +0000 (15:44 +0800)]
systemd: resolve executable path if it is relative
Backport a patch to fix the below systemd-analyze issue:
# systemd-analyze --man=false verify /lib/systemd/system/initrd-cleanup.service
initrd-cleanup.service: Command systemctl is not executable: No such file or directory
# which systemctl
/bin/systemctl
# cat /lib/systemd/system/initrd-cleanup.service
[snip]
[Unit]
Description=Cleaning Up and Shutting Down Daemons
DefaultDependencies=no
AssertPathExists=/etc/initrd-release
OnFailure=emergency.target
OnFailureJobMode=replace-irreversibly
After=initrd-root-fs.target initrd-fs.target initrd.target
Option --noline or -L does not handled properly.
So generated code contains #line directives with
file absolute path and prevents to create
reproducible builds.
Signed-off-by: Oleksiy Obitotskyy <oobitots@cisco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Barker [Tue, 19 Jan 2021 16:26:10 +0000 (16:26 +0000)]
wic: Optimise fstab modification for ext2/3/4 and msdos partitions
The fix for [Yocto #13994] required the rootfs directory to be copied
(using hardlinks if possible) when modifying the fstab file under wic.
We can optimise this copy away for filesystems where we have the tools
to modify the contents of the partition image after it is created. For
ext2/3/4 filesystems we have the debugfs tool and for msdos/vfat
filesystems we have the mcopy tool. So for any of these filesystems we
skip the modification of the fstab file in the rootfs directory (and
skip the associated copy unless it is otherwise necessary) and update
the contents of fstab directly in the partition image.
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Barker [Tue, 19 Jan 2021 16:26:09 +0000 (16:26 +0000)]
wic: Copy rootfs dir if fstab needs updating
By default, wic updates the /etc/fstab in the rootfs to include details
of additional partitions described in the selected wks file. If this
modification is performed in place, other tasks which create an image
file from the rootfs directory (e.g. do_image_tar and do_image_ext4)
will pick up the modified fstab file which would not be appropriate for
those images as they do not include the additional partitions described
in the wks file. wic does undo modifications to the fstab file once it
has finished creating the filesystem image, however this leaves open a
race condition if one of the other tasks reads the contents of the fstab
file from the rootfs directory between the point where wic modifies the
fstab file and the point where wic restores the files original content.
This could be solved by adding a lockfile for tasks which use the rootfs
directory to ensure that no other such task is reading the rootfs
directory while do_image_wic is running. This would serialize several
do_image_* tasks and result in slower builds, especially for large
images. Another drawback of this solution is that it is hard to
selectively optimise - adding lockfiles to do_image_* tasks would result
in these tasks always being serialized even if no fstab modification
will take place.
An alternative solution is to copy the rootfs directory when fstab needs
to be modified. The code to do this in wic already exists as it is
needed when including or excluding content in the rootfs. This still
results in an impact on build times but the copy uses hardlinks if
possible (so little data is actually copied) and we can make selective
optimisations to improve things. The rootfs copy will only take place if
fstab modification is required (or if it was already needed to include
or exclude rootfs content). We can also follow up with further
optimisations after this commit. So this second solution is chosen.
Fixes [Yocto #13994]
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Barker [Tue, 19 Jan 2021 16:26:08 +0000 (16:26 +0000)]
wic: Update pseudo db when excluding content from rootfs
To exclude content from the rootfs, wic makes a copy (using hardlinks if
possible) of the rootfs directory and associated pseudo db, then removes
files & directories as needed. However if these files and directories
are removed using the python functions os.remove and shutil.rmtree, the
copied pseudo db will not be updated correctly. For files copied from
the original rootfs, if hardlinks were used successfully when copying
the rootfs this should mean that the relevant inodes can't be reused and
so the risk of pseudo aborts should be avoided. However, this logic
doesn't apply for directories (as they can't be hardlinked) or for files
added via the '--include-path' argument (as they weren't present in the
original rootfs) and so there remains some risk of inodes being reused
and the pseudo db becoming corrupted.
To fix this, use the 'rm' command under pseudo when removing files &
directories from the copied rootfs to ensure that the copied pseudo db
is updated.
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Barker [Tue, 19 Jan 2021 16:26:07 +0000 (16:26 +0000)]
image_types_wic: Move wic working directory
By default the wic working directory is placed under the output
directory. When invoking wic under bitbake, the wic output directory is
added to PSEUDO_PATHS_IGNORE to avoid issues with files being removed
from outside a pseudo environment (see oe-core commit ad8f5532ff).
However, wic will copy the rootfs directory into its working directory
if it needs to add or remove content before creating a filesystem image.
This copy of the rootfs directory must be tracked by pseudo in order to
keep the permissions correct in the resulting image. So we can't have
the wic working directory under a path in PSEUDO_PATHS_IGNORE unless
we like broken permissions.
To fix this the new '-w' argument to wic is used to move the working
directory away from the output directory.
Note that wic deletes the temporary working directory automatically
when it finishes creating an image so users won't normally see the
'tmp-wic' directory under WORKDIR.
Fixes [Yocto #14129]
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Paul Barker [Tue, 19 Jan 2021 16:26:06 +0000 (16:26 +0000)]
wic: Ensure internal workdir is not reused
If a path is specified for the internal wic working directory using
the -w/--workdir argument then it must not already exist. Re-using a
previous workdir could easily result in rootfs and intermediate files
from a previous build being added to the current image.
Signed-off-by: Paul Barker <pbarker@konsulko.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
