Paul Eggleton [Tue, 3 Oct 2017 03:36:19 +0000 (16:36 +1300)]
devtool: upgrade: handle recipes that use named SRC_URI checksums
devtool upgrade did not properly handle setting SRC_URI checksums for
recipes that use named SRC_URI entries and also use those names in the
SRC_URI checksums. A further complication was where the name contained
an expression that changed with the version e.g. ${PV} (probably quite
rare, but the dnsmasq recipe in meta-networking is currently one such
recipe.) All of these are now handled properly.
Additionally, drop the _get_checksums() function that wasn't being
called from anywhere in the code.
Note that this now turns nowrap_vars in recipeutils.py to be a list of
regexes, hence things such as [ and ] need to be appropriately escaped.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Mon, 2 Oct 2017 02:45:49 +0000 (15:45 +1300)]
devtool: finish: fix "layer not in bblayers.conf" warning when path specified
devtool finish will check if the destination layer is part of
bblayers.conf so that we avoid the user getting confused about the
recipe vanishing from their configuration if it isn't. devtool finish
also accepts a path underneath a layer so that you have a bit
more control over where it ends up. However if you used a path
underneath a layer then it wasn't converting this to the base of the
layer before checking it against BBLAYERS, thus the warning was being
shown erroneously in that case.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Fri, 29 Sep 2017 03:56:20 +0000 (16:56 +1300)]
devtool: finish: ensure repository is clean before proceeding
If the git repository for a recipe in the workspace has uncommitted
changes in it then it's possible that the user has forgotten to commit
something, so check and exit if there are any. Provide a -f/--force
option to continue in the case where the uncommitted changes aren't
needed.
Separately, if the repository is in the middle of a rebase or git am /
apply then error out (without the opportunity to force) since the user
really needs to sort this out before finishing.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Fri, 29 Sep 2017 03:19:28 +0000 (16:19 +1300)]
devtool: reset: print source tree base path
If S points to a subdirectory of the source rather than the "base" of
the source tree then print that rather than the subdirectory path when
telling the user they need to remove the source tree, since that is the
directory that they will need to remove.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Fri, 29 Sep 2017 01:22:17 +0000 (14:22 +1300)]
devtool: make find-recipe and edit-recipe always work with any recipe
After some reconsideration I think it's a bit annoying for users to be
forced to use an option to work with recipes where the file isn't in the
workspace, so let's just have these commands check the workspace first
for the recipe, and if it isn't there then load the cache and get it
that way.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 28 Sep 2017 02:51:40 +0000 (15:51 +1300)]
devtool: show some warnings for upgrade versions
* Show a warning in devtool upgrade if the version is less than the
current version suggesting that the user may need to bump PE in the
recipe
* Show a warning in devtool add and devtool upgrade if the version looks
like a pre-release version suggesting using a version number that
won't mess up the progression when you come to upgrade to the final
release version.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Fri, 29 Sep 2017 02:26:22 +0000 (15:26 +1300)]
devtool: fix handling of oe-local-files when source is in a subdirectory
If S points to a subdirectory of the source rather than the "base" of
the source tree then we weren't handling the oe-local-files directory
properly - it got extracted to the base of the tree but devtool
update-recipe and devtool finish assumed it would be under S which would
be the subdirectory, thus it would be missing and devtool would assume
the files had been deleted and remove them from the recipe. Record the
base of the source tree in the bbappend and read it into the in-memory
workspace so we can use that to find out where oe-local-files should be
found.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 28 Sep 2017 03:46:14 +0000 (16:46 +1300)]
devtool: upgrade: improve performance and show progress when adding files
When devtool upgrade is upgrading to a new version where the source is
fetched as an archive (e.g. a tarball), we create a single commit in the
git repository that is the upgrade from the old version to the new. We
do this by extracting the old source, committing it, deleting all files,
copying in the new files, running git add on each new/changed/deleted
file, and then committing the result. When a lot of files have changed
in an upgrade (such as QEMU 2.8.1.1 -> 2.10.0) the penultimate step of
running git add it can take quite a long time; in order to reduce this
and show some feedback to the user, run git add with batches of 100
files at once and also show a progress bar. In a local test with the
aforementioned QEMU upgrade it took the time down from over 7 minutes
down to about 13 seconds.
Fixes [YOCTO #11948].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 8 Nov 2017 22:07:15 +0000 (11:07 +1300)]
devtool: upgrade: fix not committing deleted files with older git versions
With versions of git older than 2.0, "git add" on a deleted file (i.e.
in this case a file that was removed between versions) will not add the
delete to be committed by default, with the result that the rebase of
patches on top of the new branch will fail. We need to use the -A
option in order to force that for older git versions.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
It appears that when fixing the signature unlocking in OE-Core commit 4e9a0be32fc30fb87d65da7cd1a4015c99533aff I swapped the parameters here
and did not test it within the eSDK (it does nothing outside of the
eSDK) resulting in a TypeError when devtool upgrade was used in the
eSDK. Swap the parameters around to the correct ordering.
Fixes [YOCTO #12285].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Wed, 1 Nov 2017 00:49:44 +0000 (13:49 +1300)]
lib/oe/recipeutils: fix line splitting in patch_recipe_*
If a value was split over multiple lines (e.g. as SRC_URI usually is)
then we were inserting the value as one item in the lines list with
newlines between each line. There's nothing wrong with this if you're
writing the list out to a file, but if you want to generate a patch (as
patch_recipe_file() will do if the patch parameter is set to True) then
the diff output looks a bit odd. Split the value before adding it to the
lines list to resolve this.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Tue, 7 Nov 2017 20:45:06 +0000 (09:45 +1300)]
lib/oe/recipeutils: fix find_layerdir() to return absolute paths
find_layerdir() should really return absolute paths, so make it do so.
This fixes devtool finish not deleting files it should do after devtool
upgrade if the specified path is relative, since the devtool finish code
was assuming that find_layerdir() was returning an absolute path.
Fixes [YOCTO #12318].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
If the source tree happens to contain a kernel module as an example, a
test or under a "contrib" directory then we shouldn't be picking it up
and making the determination that the entire thing is a kernel module.
An example that triggered this is zstd, which ships a kernel module
under contrib/linux-kernel:
https://github.com/facebook/zstd
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Khem Raj [Fri, 6 Oct 2017 18:57:12 +0000 (11:57 -0700)]
musl: Update to 1.1.18
Rich Felker (9):
fix access by setjmp and longjmp to __hwcap on arm built as thumb2
for executing init array functions, use function type with prototype
fix read-after-free type error in pthread_detach
fix incorrect base name offset from nftw when pathname ends in slash(es)
in dns parsing callback, enforce MAXADDRS to preclude overflow
release 1.1.17
fix regression in glob with literal . or .. path component
fix build regression on ARM for ISA levels less than v5
release 1.1.18
Szabolcs Nagy (1):
math: rewrite fma with mostly int arithmetics
Will Dietz (1):
posix_spawn: use larger stack to cover worst-case in execvpe
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Richard Purdie [Fri, 10 Nov 2017 14:41:32 +0000 (14:41 +0000)]
testimage: Ensure full logs are shown for failures
Currently, the fact an error message is shown means the rest of the
task logs are suppressed. In this case we don't want that as it hides
the real errors and useful information. Therefore override this behaviour.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:17:17 +0000 (18:17 +0000)]
oeqa: Clean up logger handling
The logger handling in oeqa was confused at best. This patch:
a) Passes in a logger through various qemu runner pieces
b) Uses that logger consistently in the code
c) Creates a logger for QemuRunner outside the bitbake namespace
meaning we don't conflict with the tinfoil logging changes
The result of this is more consistency. For runtime tests in testimage,
the logs always contain the debug info, nothing is shwon on the console.
For the oe-selftests, logs are intercepted and only shown if the test
fails.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:15:02 +0000 (18:15 +0000)]
testimage: Pass the logger into OERuntimeTestContextExecutor.getTarget()
I have no idea why we didn't do this but it means the code has nowhere
to log to unless we do this. This means we can then use the logger
to log data to the task logs.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 18:13:24 +0000 (18:13 +0000)]
oeqa/qemurunner: Use logger.debug, not logger.info
Bitbake logs info messages to the console. These messages are really
there as debugging information. At the debug level, they will be shown
in failure logs and in the task logs but not on the console which
is what we want in this case.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Richard Purdie [Wed, 8 Nov 2017 14:03:28 +0000 (14:03 +0000)]
oeqa/runqemu: Only show stdout/stderr upon test failure
In general we don't need to see the output of runqemu however if it fails
we do. Use the buffer option that already exists in TestResult but allow
us to trigger it on a per test basis.
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Nathan Rossi [Thu, 9 Nov 2017 11:09:25 +0000 (21:09 +1000)]
busybox.inc: Fix bashism in compile, which fixes sh being suid
Fix the bashism in the suid check. This ensures that the check works
correctly on hosts that default sh to e.g. dash. If this check fails the
suid shell workaround does not remove sh from the suid binary and
results in the target system containing a busybox.suid with sh as well
as /bin/sh -> /bin/busybox.suid.
Signed-off-by: Nathan Rossi <nathan@nathanrossi.com> Cc: Andrej Valek <andrej.valek@siemens.com> Cc: Radovan Scasny <radovan.scasny@siemens.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Chen Qi [Fri, 27 Oct 2017 09:43:51 +0000 (17:43 +0800)]
gcc: fix miscompilation on mips64
We've observed strange behaviour of `systemctl status <xxx> on qemumips64.
The output of the command is like `systemctl show <xxx>', which is incorrect.
This patch is from gcc bugzilla's attachment.
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=81803
The patch hasn't been merged into gcc. But it does solve the above problem.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Markus Lehtonen [Wed, 25 Oct 2017 11:39:38 +0000 (14:39 +0300)]
oe-build-perf-report-email.py: add images as MIME objects
Add images as separate MIME objects instead of directly embedding images
in the html (as base64 encoded pngs). This makes the emails better
suited for certain email servers/clients.
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Tom Rini [Thu, 21 Sep 2017 17:46:17 +0000 (13:46 -0400)]
wic: Update canned-wks for systemd to use UUID everywhere
With systemd, the mounting of the swap partition is handled via systemd
and will mount it, regardless of if PARTUUID is parsed or not. systemd
has a runtime dependency on util-linux-mount so PARTUUID for regular
mount points will be handled correctly. Make all partitions that we add
to the image make use of UUIDs for maximum portability.
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Tom Rini [Thu, 21 Sep 2017 17:46:16 +0000 (13:46 -0400)]
wic: When using --use-uuid make sure that we update the fstab with PARTUUID
When we have been told to use the UUID we should also update the fstab
to make use of PARTUUID instead of hard-coding the device in question.
This will make the resulting image much more portable.
Signed-off-by: Tom Rini <trini@konsulko.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Chen Qi [Fri, 27 Oct 2017 09:43:50 +0000 (17:43 +0800)]
systemd: remove useless options for mips4
Looking back the history, we had problem with systemd on qemumips64
which is also related to compilation flags. We solved that by using
tweaking FULL_OPTIMIZATION for mips64 to have "-fno-tree-switch-conversion
-fno-tree-tail-merge".
Now systemd has been upgraded to 234, and we don't have the above problem
any more, thus removing these flags.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ming Liu [Mon, 2 Oct 2017 05:25:42 +0000 (07:25 +0200)]
libsolv: fix a kernel-devsrc installation issue
We encountered a problem when installing kernel-devsrc package on a
intel-x86 target, as follows:
$ dnf install kernel-devsrc
| Installing : kernel-devsrc-1.0-r0.0.intel_corei7_64 1/1
| failed loading RPMDB
| The downloaded packages were saved in cache until the next successful transaction.
| You can remove cached packages by executing 'dnf clean packages'.
It can be fixed by increasing MAX_HDR_CNT and MAX_HDR_DSIZE in libsolv
per test.
Signed-off-by: Ming Liu <liu.ming50@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Wed, 1 Nov 2017 00:28:34 +0000 (00:28 +0000)]
python: add PACKAGECONFIG for Berkeley DB module
The bsddb module is deprecated and requires an old version of Berkeley DB that
some may be unhappy with even shipping, so expose a way to disable the module.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Wed, 1 Nov 2017 13:52:19 +0000 (13:52 +0000)]
db: change types to avoid headers changing between architectures
Triggered by looking at why Python doesn't find db.h (because it greps db.h for
a regex, and db.h is actually a oe_multilib_header wrapper) I realised that the
only reason we have to oe_multilib_header db.h is because one typedef is
different between 32-bit and 64-bit architectures.
However, the typedef is for a 64-bit integer so instead of using long (64-bit)
or long long (32-bit), just use int64_t. Some of the overly complicated
configure tests need to be deleted after this change but that is safe as we're
building in a controlled environment and can assume int64_t exists.
With this done the header doesn't change between architectures, and it doesn't
need to be wrapped by oe_multilib_header.
Signed-off-by: Ross Burton <ross.burton@intel.com>
