André Draszik [Thu, 12 Jan 2017 10:34:39 +0000 (10:34 +0000)]
lib/oe/rootfs: reliably handle alternative symlinks
When removing unneeded packages from a (read-only) rootfs
during rootfs creation, alternative symlinks from those
packages may or may not be removed.
The reason is as follows:
update-alternatives(-native) is used during package
installation as part of the image creation. It uses
a database which contains entries for all the
alternative symlinks possible, and the -native version
uses the target's database by means of $OPKG_OFFLINE_ROOT,
i.e. the rootfs we're in the process of creating.
Once the rootfs has been created, OE removes certain
packages because we have a read-only rootfs - in
particular ROOTFS_RO_UNNEEDED which includes
VIRTUAL-RUNTIME_update-alternatives, i.e. the
update-alternatives. Recently, a change was made in
OE, where uninstallation of update-alternatives from the
rootfs causes removal of its database, too, to save space
(700KiB (uncompressed) in a busybox system) b24a63d71b517af701dfedbc7f7b541d25af708f
http://git.openembedded.org/openembedded-core/commit/meta/recipes-devtools/opkg-utils/opkg-utils_git.bb?id=b24a63d71b517af701dfedbc7f7b541d25af708f
Following from that, if update-alternatives is removed
from the target file system, update-alternatives-native
has no database anymore, meaning it can't manage any of
the alternative symlinks anymore.
Because the order of packages to uninstall is
non-deterministic, and update-alternatives could well
be removed before any packages that use the mechanism
provided, sometimes the extra symlinks are removed,
sometimes not.
By sorting the list of packages to be removed such that
update-alternatives is removed last, we can ensure that
that tings work reliably. (Certainly opkg seems to
uninstall packages in the order given on the command
line.)
[YOCTO #10916]
Signed-off-by: André Draszik <adraszik@tycoint.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Thu, 12 Jan 2017 12:33:59 +0000 (12:33 +0000)]
selftest/base: don't fetch DISTRO variable in constructor
Fetching the DISTRO variable in the base constructor means that we have to start
bitbake for every test case instance, which adds minutes to the startup time.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Patrick Ohly [Wed, 11 Jan 2017 11:10:33 +0000 (12:10 +0100)]
insane.bbclass: print license text as part of QA message
It it is hard to select exactly the right lines from a file, in
particular because the documentation did not specify the exact
semantic (YOCTO #10898).
When the QA license check fails, it now includes the license text for
which the md5sum was calculated. When adding a new entry to
LIC_FILES_CHKSUM, developers can then verify that they picked the
desired lines. When the checksum of an older entry changes, the developer
does not have to manually look up the changed text.
Here's an example which probably has an endline which is too large
(message triggered by changing the md5sum in the recipe):
ERROR: cmake-native-3.7.1-r0 do_populate_lic: QA Issue: cmake-native: The LIC_FILES_CHKSUM does not match for file://Source/cmake.h;beginline=1;endline=3;md5=deadbeef
cmake-native: The new md5 checksum is 4494dee184212fc89c469c3acd555a14
cmake-native: Here is the selected license text:
vvvvvvvvvvvvvvvvvvvvvvvvvvvv beginline=1 vvvvvvvvvvvvvvvvvvvvvvvvvvvvv
/* Distributed under the OSI-approved BSD 3-Clause License. See accompanying
file Copyright.txt or https://cmake.org/licensing for details. */
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ endline=3 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
cmake-native: Check if the license information has changed in .../cmake.h (lines 1 through to 3) to verify that the LICENSE value "BSD" remains valid [license-checksum]
The beginline/endline values are only repeated in the borders if set.
License snippets larger larger than 20 lines (configurable with
QA_MAX_LICENSE_LINES) are truncated in the middle.
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Alistair Francis [Tue, 10 Jan 2017 21:04:00 +0000 (13:04 -0800)]
runqemu: Allow the user to specity no kernel or rootFS
In some cirsumstances the user doesn't want to supply a kernel, rootFS
or DTB to QEMU. This will occur more now that QEMU supports loading
images using a '-device loader' method.
Allow users to specify 'none' for QB_DEFAULT_FSTYPE or QB_DEFAULT_KERNEL
to avoid supplying these options to QEMU.
Signed-off-by: Alistair Francis <alistair.francis@xilinx.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Tue, 10 Jan 2017 17:46:09 +0000 (19:46 +0200)]
wic: _exec_cmd: produce error if exit code is not 0
Current code doesn't always show error output of the
external command and even ignores non-zero exit code.
Moved checking of exit code value to the lowest level
possible: to _exec_cmd. This should make wic to always
check exit code of the external command and issue
an error if it's not 0.
[YOCTO #10816]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ed Bartosh [Tue, 10 Jan 2017 17:46:08 +0000 (19:46 +0200)]
direct.py: fix getting image name
part.rootfs_dir was used as an image name in the code.
However, when multi-rootfs feature is used this attribute
points to the name of the rootfs, e.g. if --rootfs command line
is rootfs1=core-image-minimal partf.rootfs_dir is 'rootfs1'.
The code also fails when image name is not provided in wic
commandline. For example, when wic is called with
--rootfs-dir=<path> part.rootfs_dir will contain path and
wic will crash trying to call bitbake -e <path> to get
value of ROOTFS_SIZE variable.
Fixed the code by getting image name properly and checking
if it's not a path.
[YOCTO #10815]
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Maxin B. John [Tue, 10 Jan 2017 12:18:38 +0000 (14:18 +0200)]
sqlite3: upgrade to 3.16.2
3.15.2 -> 3.16.2
1. Updated the SRC_URI for releases in 2017
2. Removed the following revert patch as the fix is present in this release:
a) 0001-revert-ad601c7962-that-brings-2-increase-of-build-ti.patch
[YOCTO #10695]
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Tue, 10 Jan 2017 15:09:48 +0000 (15:09 +0000)]
rootfs: don't put /usr/lib/ssl and /etc into debugfs
The /etc and /usr/lib/ssl directories were only put into the opkg-generated
debugfs because of a bug in opkg which means that a conffile has to exist if
we're running 'opkg status'. This is now fixed, so the workaround can be
reverted.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Tue, 10 Jan 2017 15:39:47 +0000 (15:39 +0000)]
opkg: fix conffile errors in 'opkg status' calls
If a conffile has been deleted (common when building a debugfs) the status
command will throw errors instead of handling that situation. Stop the code
being executed in the first place if it wasn't asked for, and handle errors
gracefully.
[ YOCTO #10761 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Ola x Nilsson [Mon, 9 Jan 2017 16:44:57 +0000 (17:44 +0100)]
externalsrc.bbclass: Add task buildclean
The buildclean task should call the package build system clean
command, just implemented for Make for now.
This is meant for recipes where S == B, but can be useful as a
standalone task for other recipes too.
When S == B, set it to run before do_clean which will do what most
developers expect when calling bitbake -c clean. For S != B, do not
add it before clean as it is not needed and may take some time.
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
The purpouse of this function was to check dependencies for building a
hybrid iso and build them using bitbake if not found. Calling bitbake in
this context means this wic plugin itself cannot be instrumented inside
bitbake recipes which is undesirable, the benefits of this are clear:
there is no need to maintain outside scripts to generate an iso using wic
and the isohybrid building logic can be further abstracted away into an
isohybrid.bbclass in the future which can be easily inherited or something
similar.
So remove the function and add all dependencies to NATIVE_RECIPES so that
wic can print useful errors when they're not built.
To automate building the isohybrid image dependencies, add the following
somewhere in your image build inheritence hierarcy (or maybe create a
bbclass in the future to do these sort of things automatically):
Patrick Ohly [Mon, 9 Jan 2017 10:09:42 +0000 (11:09 +0100)]
rootfs-postcommands.bbclass: sort passwd entries
The /etc passwd files in a rootfs consist of the default entries from
base-passwd plus anything that gets added via package installation,
EXTRA_USERS_PARAMS and/or system sysusers.
The execution order of preinst scripts is not perfectly deterministic,
or at least unrelated changes caused it to change in a
non-deterministic way, resulting in irrelevant changes in the order of
passwd entries.
useradd-staticids.bbclass ensures that the numeric IDs don't change,
but re-ordering can still occur, which is bad for reproducible builds
and file-based update mechanisms like swupd which work best if changes
are as minimal as possible.
To achieve that, the files get sorted in a post-processing command,
enabled by default. Sorting is based primarily on the numeric IDs, so
for example, the "root" user continues to be listed first. "nobody"
now is at the end, which wasn't the case before.
The order of the entries should not matter, but in obscure cases where
it does (like having multiple entries for the same numeric ID) this
behavior can be disabled by setting SORT_PASSWD_POSTPROCESS_COMMAND to
an empty string.
Fixes: YOCTO #10520 Signed-off-by: Patrick Ohly <patrick.ohly@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Mon, 9 Jan 2017 15:28:10 +0000 (15:28 +0000)]
flex: upgrade to 2.6.2
Patches dropped as they are merged upstream:
- CVE-2016-6354.patch
- 0002-avoid-c-comments-in-c-code-fails-with-gcc-6.patch
- do_not_create_pdf_doc.patch
Apply a patch from github to simplify cross-compilation and not need a
flex-native to bootstrap.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Mark Asselstine [Tue, 3 Jan 2017 17:01:58 +0000 (12:01 -0500)]
sysklogd: do more to properly work with systemd
It was noticed that syslogd and klogd were no longer running on system
startup, meaning no /var/log/messages etc.. It appears as though
sysklogd has never been updated to follow the expected logging
requirement for systemd as described here:
https://www.freedesktop.org/wiki/Software/systemd/syslog/
As such no service was started and no logging present. Using the above
guidelines we create two new service files syslogd.service and
klogd.service. We make use of tmpfiles.d in order to ensure the
xconsole device node exists and do other minor recipe cleanup to
ensure peaceful coexistence with sysvinit and systemd implementations.
The systemd documentation also asks that for a logger which is not
rsyslog that we also enable 'ForwardToSyslog=' in journald.conf, but
this is already the case so no action is required.
With this change in place syslogd and klogd are started at system
startup and the expected logs are available.
Unfortunately I was not able to find any work done on this upstream or
in other distros so this is my best effort at making this work.
Signed-off-by: Mark Asselstine <mark.asselstine@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Juro Bystricky [Fri, 30 Dec 2016 21:35:32 +0000 (13:35 -0800)]
build-appliance-image: support for Toaster
Various changes needed to enable to run Toaster in the
Build Appliance:
1. Pre-install packages as specified by the file
"bitbake/toaster-requirements.txt"
2. Include pip3 in the image
3. Include tzdata in the image (needed by django)
4. Bump SRCREV to a commit with proper settings.py (ALLOWED_HOSTS)
for Django 1.8.16
5. Added README_VirtualBox_Toaster.txt to provide steps for
configuring VirtualBox network adapters (NAT or Bridged)
and steps to launch Toaster
[YOCTO#10767]
Signed-off-by: Juro Bystricky <juro.bystricky@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Randy Witt [Thu, 5 Jan 2017 23:15:44 +0000 (15:15 -0800)]
image_types.bbclass: IMAGE_TYPEDEP_ now adds deps for conversion types
Previously if IMAGE_TYPEDEP_* contained a conversion type of the form,
"foo.bar", the dependency on CONVERSION_DEPENDS_bar would not get added
to the task depends for do_rootfs.
[YOCTO #10883]
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Randy Witt [Thu, 5 Jan 2017 23:15:43 +0000 (15:15 -0800)]
image_typedep.py: Add a test that ensures conversion type deps get added
Add a test that ensures if IMAGE_TYPEDEP_* contains a conversion type,
that the corresponding CONVERSION_DEPENDS_ for that type gets added to
the dependency tree for do_rootfs.
Signed-off-by: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ross Burton [Thu, 22 Dec 2016 17:07:43 +0000 (17:07 +0000)]
linux-firmware: remove alternatives for brcmfmac-stdio.bin
These alternatives are only used to provide an unversioned brcmfmac-sdio.bin,
which was required by kernels prior to 3.13. As these alternatives all have the
same priority there's no determinism in which one is selected, and current
kernels (since January 2014) use the appropriately versioned firmware names.
Signed-off-by: Ross Burton <ross.burton@intel.com>
gummiboot: Remove old gummiboot recipe, related class and wks file
Since the gummiboot project is no longer being maintained
and we are using systemd-boot as a replacement instead,
we can now clean up all remaining gummiboot files.
[YOCTO #10332]
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Robert Yang [Tue, 20 Dec 2016 03:09:45 +0000 (03:09 +0000)]
hdparm: 9.48 -> 9.50
Remove "-e MAKEFLAGS=" from EXTRA_OEMAKE to fix:
ERROR: hdparm-9.50-r0 do_package: QA Issue: File '/sbin/hdparm.hdparm' from hdparm was already stripped, this will prevent future debugging! [already-stripped]
ERROR: hdparm-9.50-r0 do_package: Fatal QA errors found, failing task.
The "-e MAKEFLAGS=" would cause submake can't get vars from environment,
The git log said that it was added for fixing a QA warning, but
everything is OK after remove it now.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
This is the usual way this is handled in desktop distros (see debian, gentoo).
I wasn't able to track down a patch to add proper x32 support to ffmpeg. There
was, however, a libav patch series which may be worth investigating.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
boost was adding -march/-mcpu itself, and adding -m32/-m64 itself as well.
Patch that behavior out, apply another similar patch from elsewhere, and
adjust BJAM_OPTS to fix the x32 build.
Signed-off-by: Christopher Larson <chris_larson@mentor.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mike Crowe [Wed, 4 Jan 2017 19:28:46 +0000 (19:28 +0000)]
mtd-utils: Upgrade to 2.0.0
Upstream has started using automake which means that the recipe must now
inherit from autotools and pkgconfig.
The source tree has been reorganised too which requires the paths in the
patches to be modified. None of the patches appear to have been applied
upstream.
Signed-off-by: Mike Crowe <mac@mcrowe.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Haiqing Bai [Thu, 5 Jan 2017 03:32:54 +0000 (11:32 +0800)]
nettle: Modify configure script to get consistent build.
The original configure script detects the header files
of openssl to set variable like 'HAVE_OPENSSL_AES_H' in
config.h and ignore the value of '--enable-openssl', this
may cause inconsistent build.
Signed-off-by: Haiqing Bai <Haiqing.Bai@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
David Vincent [Wed, 4 Jan 2017 16:43:02 +0000 (17:43 +0100)]
nss: Fix postinstall script
When installing NSS on a read-only rootfs, the current postinstall
scriptlet exits after having run the signing part. This causes an error
when appending the task because the rest of the script is simply ignored
and therefore never run.
Signed-off-by: David Vincent <freesilicon@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Khem Raj [Wed, 21 Dec 2016 17:34:12 +0000 (09:34 -0800)]
opkg-utils: Do not use --ignore-fail-on-non-empty with rmdir
--ignore-fail-on-non-empty is coreutils specific, and you
are not always going to have coreutils on target systems
especially small ones. They will use the busybox applet which
does not support --ignore-fail-on-non-empty, use pipe and true
to ignore the errorcode from rmdir instead
Ola x Nilsson [Thu, 22 Dec 2016 13:16:33 +0000 (14:16 +0100)]
oe-selftest: devtool: Reverting a change should trigger rebuild
Add code to verify that not only does a change trigger a build, but so
does reverting that change.
Reverting a change in a devtool managed git repo may cause the current
checksum to match the checksum of a previous build, which will cause
bitbake to skip builds that are needed.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Jianxun Zhang [Wed, 21 Dec 2016 22:31:46 +0000 (14:31 -0800)]
lib/oe.sstatesig: make locked sig file consistent
Sort keys of dict 'types' prior to dumping, in order to have
identical output every time. This could make it a little easier
to diff these human-readable dumps.
Signed-off-by: Jianxun Zhang <jianxun.zhang@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 22 Dec 2016 03:13:56 +0000 (16:13 +1300)]
meta: use require instead of include when file should exist
If the file is expected to exist, then we should always be using require
so that if it doesn't we get an error rather than some other more
obscure failure later on.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 22 Dec 2016 03:13:57 +0000 (16:13 +1300)]
oe-selftest: fix behaviour if oe-selftest.log is a dangling symlink
If you delete the log file that the oe-selftest.log symlink points to
but not the symlink itself, because we were using os.path.exists() here
the code assumed that the symlink didn't exist when in fact it still
did. Use os.path.lexists() instead.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
gummiboot: Remove/change gummiboot references with systemd-boot
After systemd-boot was introduced, its been tested for a while with no major
issues being found until now, this patch completely replaces all gummiboot
instances with systemd-boot ones, taking the next step into cleaning
up systemd-boot/gummiboot.
[YOCTO #10332]
Signed-off-by: Alejandro Hernandez <alejandro.hernandez@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Chen Qi [Wed, 21 Dec 2016 04:32:47 +0000 (12:32 +0800)]
opkg-utils: warn if update-alternatives finds priority conflict
If multiple providers for a utility have the same alternatives priority,
which one would be chosen is determined by which one is installed later.
Our alternatives system should be able to detect such problem and warn users
so that potential problems could be avoided.
Modify update-alternatives to warn users when detecting multiple providers
with the same priority.
David Vincent [Tue, 20 Dec 2016 09:47:45 +0000 (10:47 +0100)]
classes: Fix alternatives and rc.d ordering
When using an alternative as an initscript, the ordering between
update-rc.d and update-alternatives tasks during prerm and postinst
tasks must always be the following in order to work:
* prerm:
- stop daemon
- remove alternative
* postinst:
- add alternative
- start daemon
This patchset adds comments to the scripts generated by both classes and
organize the generated sections based on those comments.
[YOCTO #10433]
Changes since v5:
- Remove boolean in d.getVar() calls
Signed-off-by: David Vincent <freesilicon@gmail.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ola x Nilsson [Thu, 22 Dec 2016 13:16:34 +0000 (14:16 +0100)]
externalsrc: Set STAMPCLEAN to match STAMP
Only the last stamp file should be kept, but unless STAMPCLEAN matches
files generated using STAMP old stamp files may linger. This may
cause false positives for skipping tasks.
Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 22 Dec 2016 02:19:59 +0000 (15:19 +1300)]
classes/sstate: fix file conflict message
* Fix not decoding output from grep ("Matched in b'manifest...')
* Fix showing "Matched in b''" if no match (show "not matched to any
task" instead)
* Drop the filtering out of .populate-sysroot from matched manifest
names - it should have been .populate_sysroot so it doesn't work, and
in any case the value of removing the task name is questionable given
that we aren't removing it for any other task, and that the rest of
the filename isn't only the task name, we might as well have the whole
thing. At least then you can do a find on that exact name without
wildcards and find it.
* Fix indenting of file list entries and indent "matched in" further
underneath
* Minor punctuation fixes
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
If a recipe installs a file or directory whose name contains square
brackets [ ] that form a valid glob expression and that file then they
won't be correctly removed from the sysroot, because we pass each path
in the sstate manifest to our oe.path.remove() function which calls
glob.glob() on the path passed into it and the expression won't
actually match the original filename. Since we don't expect to put any
wildcarded expressions in the sstate manifests, and we already have a
try...except around this, we can actually use os.remove() here instead.
Similarly, when we pass existing file paths to "grep" looking through
the manifests, we don't want those paths to be treated as regexes - so
use grep's -F command line switch.
Fixes [YOCTO #10836].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
