Nathan Rossi [Fri, 16 Jun 2017 12:38:49 +0000 (22:38 +1000)]
kernel-uimage.bbclass: Fix up generation of uImage from vmlinux
Fix up the generation of uImage from vmlinux when KEEPUIMAGE != 'yes'.
This fixes up the working directory that do_uboot_mkimage is run from,
such that it is run from the ${B} directory to access built artefacts.
Simplify the logic in the task so that the parse step either adds the
task or not if the conditions are met. This reduces the need for the
task to run in cases when it is not used. The task is also changed to
depend on the kernel_link_images task as arch/<arch>/boot/* is not
available until after kernel_link_images in certain cases (e.g.
vmlinux/uImage only KERNEL_IMAGETYPES).
Fix up the use of ${S}/vmlinux when pulling the entry symbols
address so that it accesses the vmlinux in ${B}.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Petter Mabäcker [Mon, 15 May 2017 04:17:11 +0000 (06:17 +0200)]
multilib.bbclass: fix faulty redefinition of STAGING_KERNEL_DIR
Due to the problem fixed in
'56c677a multilib: Move redefinition of STAGING_DIR_KERNEL'
STAGING_KERNEL_DIR must be redefined for lib32 in multilib.bbclass.
However this redefinition expanded STAGING_KERNEL_DIR to an absolute
path. This unconsciously added the TMPDIR path in the sstate object,
causing packages depended on STAGING_KERNEL_DIR being rebuild if the
TMPDIR was changed.
Solve this by forcing the unexpanded TMPDIR variable to remain in the
beginning of STAGING_DIR_KERNEL (as default). Since TMPDIR is included in
BB_HASHBASE_WHITELIST, the sstate object will not be depended on the
expanded path anymore.
Signed-off-by: Petter Mabäcker <petter@technux.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
Yeoh Ee Peng [Wed, 14 Jun 2017 20:01:12 +0000 (13:01 -0700)]
selftest/devtool: replace file assertTrue/False with assertExists/NotExists
Current osselftest print confusing assertion message when using
self.assertTrue(os.path.exists(filepath)) to test file path,
example of confusing assertion message:
AssertionError: False is not true
Replce assertTrue/assertFalse with assertExists/assertNotExists to test
file path, this will improve assertion message and simplify coding,
self.assertExists(filepath) will print below
AssertionError: <filepath> does not exist
[YOCTO #11356]
Signed-off-by: Yeoh Ee Peng <ee.peng.yeoh@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Fri, 16 Jun 2017 13:19:30 +0000 (16:19 +0300)]
wic: fix restoring of original fstab
Wic updates fstab if mount points are specified in .wks.
After partition images are created the original fstab is restored.
However, if exception is raised when partition image being prepared
wic doesn't restore original fstab. This can cause duplication of
added lines in fstab when 'wic create' runs next time.
Wrapping call of 'prepare' method and restoring original fstab
even if exception occurs should fix this.
[YOCTO #11633]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
scripts/contrib/patchtest: run patchtest on local branch
The script run patchtest on local branch commits, printing results into
stdout. This script is useful to test patches before sending to the
mailing list.
Examples:
$ git checkout master-next-1.9
Branch master-next-1.9 set up to track remote branch master-next-1.9 from origin.
Switched to a new branch 'master-next-1.9'
$ ~/scripts/contrib/patchtest.sh 166e70e: Robert Yang: Thu Apr 2 12:01:37 2015 +0100: patch: fix CVE-2015-1196: FAIL
Issue Missing or incorrectly formatted CVE tag in commit message [test_cve_presence_in_commit_message]
Suggested fix Include a "CVE-xxxx-xxxx" tag in the commit message
Issue Missing or incorrectly formatted CVE tag in included patch file [test_cve_tag_format]
Suggested fix Correct or include the CVE tag on cve patch with format: "CVE: CVE-YYYY-XXXX"
eaa4536: Robert Yang: Thu Apr 2 12:01:37 2015 +0100: wget: 1.16.1 -> 1.16.2: OK
3c29ce3: Robert Yang: Thu Apr 2 12:01:38 2015 +0100: git: 2.3.0 -> 2.3.1: OK
It's possible for tasks to stage symlinks that point to non-existent
files; an example is ncurses-native.do_populate_sysroot. There wasn't
any error checking here so this broke the build when "task" was included
in BUILDHISTORY_FEATURES. In any case we shouldn't be following symlinks
and getting the sha256sum of the link target - we need concern ourselves
only with the target path, so check if the file is a link and sha256 the
target path instead if it is. If it's neither a regular file nor a
symlink (perhaps a pipe or a device), just skip it.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 15 Jun 2017 22:43:22 +0000 (23:43 +0100)]
pciutils: Fold patches into a single patch with a description
Patches patching content from other patches is bad. This folds the three
patches into one, improves some of what its doing to be a little
cleaner and adds a proper description to the patch header.
This also moves the STRIP override to the configure commandline rather
than patching and handles passing libdir the same way.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 15 Jun 2017 17:25:54 +0000 (18:25 +0100)]
meta: Drop further remnants of uclibc support
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.
uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Thu, 15 Jun 2017 14:21:42 +0000 (15:21 +0100)]
meta: Drop remnants of uclibc support
uclibc support was removed a while ago and musl works much better. Start to
remove the various overrides and patches related to uclibc which are no longer
needed.
uclibc support in a layer would still be possible. I have strong reasons to
believe nobody is still using uclibc since patches are missing and I doubt
the metadata even parses anymore.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 25 Apr 2017 14:25:54 +0000 (15:25 +0100)]
pseudo: Handle too many files deadlock
If we have large amounts of parallelism, pseudo can end up with too
many open connections and will no longer accept further connections,
hanging. This patch works around that by closing some clients, allowing
turnover of connections and unblocking the system. The downside is a small
but theoretical window of data loss. This is likely better than locking
up entirely though. Discussions with Peter are onging about how we could
better fix this.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The recent ovmf update broke secureboot because upstream changed the
way how openssl gets compiled into ovmf. It's now integrated directly
into the ovmf build process, without having to patch it first.
In addition, more recent OpenSSL releases are supported. 1.1.0e was
explicitly mentioned in the ovmf commits and because the current
1.1.0f only has minor build enhancements, 1.1.0e is used here.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Fri, 16 Jun 2017 08:42:30 +0000 (09:42 +0100)]
package_ipk: Clean up Source entry in ipk packages
There is the potential for sensitive information to leak through the urls
there and removing it brings this into the behavior of the other package
backends since filtering it is likely error prone.
Since ipks don't appear to be generated at all if we don't set this, set
the field to the recipe name used (basename only, no paths). This avoids
information leaking. We may want to drop the field if opkg can allow that
at a future point but the recipe name is a suitable identifier for now.
Reported-by: Andrej Valek <andrej.valek@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
package_manager: don't race on a file when installing complementary
packages
the file isn't closed before oe-pkgdata-util uses it and this
temporary file might look empty to oe-pkgdata-util, because it
wasn't flushed yet. Which resulted in almost empty debugfs tarballs
and no locale packages in regular rootfs.
* without this change:
124K May 30 07:41 core-image-full-cmdline-raspberrypi3-64-20170530054003-dbg.rootfs.tar.gz
* with this change:
173M May 30 07:29 core-image-full-cmdline-raspberrypi3-64-20170530052715-dbg.rootfs.tar.gz
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Patrick Ohly [Wed, 14 Jun 2017 10:33:50 +0000 (12:33 +0200)]
distrooverrides.bbclass: DISTRO_FEATURES as overrides
This achieves the same goal as the same change to bitbake.conf itself,
but because the class gets added later as part expanding INHERIT, this
new approach is less likely to run into problems when DISTRO_FEATURES
contains complex code.
Another difference is that the class currently does not get inherited
by default and thus is completely absent from a build unless some
layer or include file adds it to INHERIT.
Compared to the earlier code in bitbake.conf and a similar class in
intel-iot-refkit, additional overrides now get sorted. This makes the
final OVERRIDES more deterministic.
The lessons learned about unintentionally depending on OVERRIDES are
documented in the class because such problems are more likely to show
up as unexpected signature differences when using this class.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
It turned out that the code which expands DISTRO_FEATURES early during
base config parsing can fail because some entries in DISTRO_FEATURES
might call Python functions like base_conditional() from base.bbclass
which aren't defined yet.
A different solution will be needed.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Wed, 14 Jun 2017 10:25:38 +0000 (12:25 +0200)]
python-3.5: Move bz2.py, lzma.py and _compression.py from python3-misc to python3-compression
* the /usr/lib/python3.5/_compression.py file is possibly incorrectly included
in python3-misc. This runtime dependency is needed in order to use e.g. gzip.py in runtime:
>>> import tarfile, zlib, gzip
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.5/gzip.py", line 12, in <module>
import _compression
ImportError: No module named '_compression'
* at least python3-tests and lzma and bz2 still in python3-misc are using this as well:
$ grep -R import.*_compression tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-compression/usr/lib/python3.5/gzip.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/lzma.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-misc/usr/lib/python3.5/bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_bz2.py:import _compression
tmp-glibc/work/aarch64-oe-linux/python3/3.5.2-r1.0/packages-split/python3-tests/usr/lib/python3.5/test/test_lzma.py:import _compression
and python3-tests are using it as well, so add new runtime dependency
on python3-compression
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mark Asselstine [Tue, 13 Jun 2017 20:26:45 +0000 (16:26 -0400)]
attr: Fix build failure when image includes man-pages pkg
If you attempt to build an image with both attr(-doc) and man-pages
packages your rootfs might fail to assemble. The error will be
something like:
Error: Transaction check error:
file /usr/share/man/man2/fgetxattr.2 from install of \
attr-doc-2.4.47-r0.core2_64 conflicts with file from \
package man-pages-4.11-r0.core2_64
(the error is usually only seen on builders which don't have manpages
installed, if you have /usr/share/man/man1/man.1.gz your build will
complete but you will have duplicate manpages, just one zipped and one
not)
Backporting changes from upstream attr removes the conflicted files in
favour of those in the man-pages package.
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
adwaita-icon-theme: add a patch to speed up the do_install() task
Goes down to 40 seconds from over 4 minutes :)
Note that there is no control over the amount of shell jobs; on my machine
this is not a problem, but if it's a problem on less capable hardware,
we can add some kind of limiter.
Signed-off-by: Alexander Kanavin <alexander.kanavin@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mikko Ylinen [Tue, 13 Jun 2017 15:39:09 +0000 (18:39 +0300)]
runqemu: change terminal settings for valid tty's
runqemu uses stty to change terminal settings to give users
better control to qemu. However, stty does not work when
runqemu is run directly or indirectly via oe-selftest in
a Docker container (presumably some problems with Docker's
pseudo-tty implementation).
The error reported is:
stty: 'standard input': Inappropriate ioctl for device
As runqemu recently moved to subprocess.check_call() for
stty calls we now get thrown an error and all runqemu
runs fail.
sys.stdin.isatty() does proper job in detecting if the stty
calls can work so we use that check before running the stty
subprocess operations.
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
uboot-config: check UBOOT_CONFIG variable, not flags, for error conditions
Sometimes there's a need to change existing UBOOT_CONFIG setting from a recipe,
distro or local config, such as an override or even switch back to UBOOT_MACHINE.
Unfortunately, there's no easy way to override or unset flags, so using them as
an error condition is rather heavy-handed. Change those conditions to check the
UBOOT_CONFIG variable itself, not its flags.
Signed-off-by: Denys Dmytriyenko <denys@ti.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Amarnath Valluri [Tue, 13 Jun 2017 06:52:38 +0000 (09:52 +0300)]
sed: changes to support merged /usr
Few of the perl scripts referring '#!/bin/sed' inside the script. But when
'usrmerge' feature is enabled this path would be /usr/bin/sed. So to satisfy
build dependency add '/bin/sed' to it's providers list.
Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Amarnath Valluri [Tue, 13 Jun 2017 06:52:37 +0000 (09:52 +0300)]
busybox: changes to support merged /usr
Most of the shell scripts refer to /bin/sh inside the script. When 'usrmege'
feature is enabled, this path would be /usr/bin/sh. Hence, to satisfy build
dependency add '/bin/sh' to it's providers list.
Signed-off-by: Amarnath Valluri <amarnath.valluri@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Mike Crowe [Tue, 16 May 2017 12:40:00 +0000 (13:40 +0100)]
gcc-cross-canadian: Use ${target_includedir} for compatibility with meta-micro
meta-micro puts headers in /include rather than /usr/include in the
sysroot. ${target_includedir} means that the correct path will be used
automatically.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Acked-by: Phil Blundell <pb@pbcl.net> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:22:10 +0000 (14:22 +0300)]
wic: add help and usage content for 'wic rm'
Added wic_rm_help and wic_rm_usage variables to help.py.
These variables contain help content that will be used in
'wic rm help' and 'wic rm --help' output.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:22:06 +0000 (14:22 +0300)]
wic: add Disk._put_part_image method
This method copies partition image into the wic image.
It will be used in 'wic cp' and 'wic rm' subcommands
to copy changed partition back into wic image.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:22:05 +0000 (14:22 +0300)]
filemap: calculate dst size correctly
Fixed calculation of the dst file size using skip, seek and
length parameters. Current code does it incorrectly which
causes sparse_copy API to create unnecessary big output files.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:22:04 +0000 (14:22 +0300)]
filemap: check if dest is written for every block
If lenght parameter is provided to sparse_copy call
it's mandatory to check if the output file is fully
written after reading unmapped block from input file.
If it's not done then sparse_copy can write more data
than specified length.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:21:59 +0000 (14:21 +0300)]
wic: add help and usage content for 'wic cp'
Added wic_cp_help and wic_cp_usage variables to
help.py. These variables contain help content that
will be used in 'wic cp help' and 'wic cp --help'
output.
[YOCTO #11283]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:21:54 +0000 (14:21 +0300)]
wic: add help and usage content for 'wic ls'
Added wic_ls_help and wic_ls_usage variables to
help.py. These variables contain help content that
will be used in 'wic ls help' and 'wic ls --help'
output.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:21:52 +0000 (14:21 +0300)]
bootimg-pcbios: make boot image file unique
Plugin code uses boot.img file name for an image file. If there are
two partitions that use bootimg-pcbios wic breaks with an error
"file already exists: boot.img"
Made image file name unique by adding wks like number to it to fix
the issue.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:21:51 +0000 (14:21 +0300)]
filemap: add parameter 'length' to sparse_copy
Added parameter 'length' to specify amount of data
to write into destination file. This is useful when only
part of source file should be written into destination file.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ed Bartosh [Tue, 13 Jun 2017 11:55:52 +0000 (14:55 +0300)]
mtools-native: disable reading host configs
Removed code that reads /etc/mtools.conf, /etc/default/mtools.conf,
/etc/mtools and /etc/default/mtools to ensure that mtools output
doesn't depend on the global host configs.
It's still possible to use ~/.mtoolsrc config or point MTOOLSRC
environment variable to any configuration file if user want
to configure mtools.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 13 Jun 2017 09:22:28 +0000 (10:22 +0100)]
bitbake.conf: Don't exclude MACHINE/MACHINEOVERRIDES from hashes
A long time ago (6 years), this seemed like a good idea. The reality is
that OVERRIDES should not be being added to hashes and if it is, it likely
needs excluding in its own right. This was a nice workaround but we need
to fix the real underlying issues now. In some cases this means excluding
OVERRIDES from the variables dependency using the vardepsexclude flag however
caution is needed to ensure this is safe.
Variable values used to construct hashes are unexpanded but the values used
are computed after the application of OVERRIDES. The important detail is if
the end resulting unexpanded value changes, not the value of the OVERRIDES
used in the construction of that unexpanded value. This is why dependencies
on OVERRIDES itself shouldn't be in the hashes in general.
The recent DISTRO_FEATURES changes adding in override mappings for them
highlighted this issue. We have some good sstate tests which are effective
at highlighting where potential issues arrive with OVERRIDES contamination
(oe-selftest -r sstatetests.SStateTests).
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Tue, 13 Jun 2017 09:14:15 +0000 (10:14 +0100)]
insane: Don't depend on OVERRIDES
In common with the other package handling functions, don't depend on the
value of OVERRIDES. This means when we change MACHINE, we don't have to
repackage everything.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Mon, 12 Jun 2017 21:41:18 +0000 (16:41 -0500)]
oeqa/core/loader: Use full and small module name on filtering
The small module name was added to support run a whole suite that
has more that 3 levels in the test case name, but this broke the
behaviour for use a full test case name.
[YOCTO #11632]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Ming Liu [Mon, 12 Jun 2017 07:25:52 +0000 (09:25 +0200)]
rpm: add explicit dependency on bzip2-replacement-native for native builds
This fixes a following error:
| recipe-sysroot-native/usr/lib/rpm/debugedit: error while loading shared libraries: \
| libbz2.so.1: cannot open shared object file: No such file or directory
Signed-off-by: Ming Liu <peter.x.liu@external.atlascopco.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
