Ross Burton [Tue, 6 Jun 2017 14:23:18 +0000 (15:23 +0100)]
archiver: preserve sysroot paths in configured mode
do_ar_configured alters WORKDIR but also expects to be able to run do_configure,
so forcibly expand the paths to the sysroots as otherwise they'll point to a
non-existant directory in the temporary WORKDIR.
[ YOCTO #11584 ]
Signed-off-by: Ross Burton <ross.burton@intel.com>
Dengke Du [Mon, 28 Nov 2016 12:37:35 +0000 (12:37 +0000)]
archiver.bbclass: fix do_ar_original error for matchbox-desktop
Error:
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ERROR: matchbox-desktop-2.1-r0 do_ar_original: Can not determine archive names
for original source because 'name' URL parameter is unset in more than one URL.
Add it to at least one of these: git://git.yoctoproject.org/matchbox-desktop-2
file://vfolders/%2A
ERROR: matchbox-desktop-2.1-r0 do_ar_original: Function failed: do_ar_original
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In function do_ar_original, when recipes have more than one source, it added the
"name" URL parameter as suffix to identify the created tarball.
But the URL type "file://" that we always used to represent a series of patches,
it didn't have "name" parameter, so it failed.
So set "name" to the folder name to identify the created tarball, for example:
In matchbox-desktop bb file, the SRC_URI contains:
file://vfloders/*
We set "name" to "vfolders" to identify the created tarball.
In connman-gnome bb file, the SRC_URI contains:
file://images/*
We set "name" to "images" to identify the created tarball.
Signed-off-by: Dengke Du <dengke.du@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mikko Ylinen [Mon, 5 Jun 2017 14:42:51 +0000 (17:42 +0300)]
archiver.bbclass: adapt do_unpack_and_patch to RSS
do_unpack_and_patch was not correctly run until recently
("archiver.bbclass: various fixes for original+diff mode") but
with the fix applied, the errors we get indicate the function
is not adapted to work with recipe specific sysroots.
do_unpack_and_patch sets WORKDIR to ARCHIVER_WORKDIR which
affects all path settings relative to WORKDIR, inluding the paths
to recipes' sysroots. IOW, when do_unpack and do_patch are run, they
cannot find the necessary native tools and files located in the
sysroot (e.g., quiltrc) because the paths point to ARCHIVER_WORKDIR.
Adapt do_unpack_and_patch to RSS by restoring the original
STAGING_DIR_NATIVE after WORKDIR is changed to ARCHIVER_WORKDIR.
Signed-off-by: Mikko Ylinen <mikko.ylinen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Anibal Limon [Mon, 5 Jun 2017 08:39:08 +0000 (04:39 -0400)]
oeqa/core/loader: Call parent init method in OETestLoader
After Python 3.5 the init method contains initialization of
_loading_packages set to avoid infinite loops in recursive
loading.
Fix,
Traceback (most recent call last):
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 70, in <module>
ret = main()
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/scripts/oe-selftest", line 57, in main
results = args.func(logger, args)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 214, in run
rc = self._internal_run(logger, args)
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/selftest/context.py", line 175, in _internal_run
self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/context.py", line 51, in loadTests
self.suites = self.loader.discover()
File "/home/pokybuild/yocto-autobuilder/yocto-worker/nightly-oe-selftest/build/meta/lib/oeqa/core/loader.py", line 268, in discover
pattern='*.py', top_level_dir=path)
File "/usr/lib/python3.5/unittest/loader.py", line 341, in discover
tests = list(self._find_tests(start_dir, pattern))
File "/usr/lib/python3.5/unittest/loader.py", line 398, in _find_tests
full_path, pattern, namespace)
File "/usr/lib/python3.5/unittest/loader.py", line 473, in _find_test_path
self._loading_packages.add(name)
AttributeError: 'OETestLoader' object has no attribute '_loading_packages'
Signed-off-by: Anibal Limon <anibal.limon@linux.intel.com>
Khem Raj [Fri, 2 Jun 2017 16:01:50 +0000 (09:01 -0700)]
distutils-base.bbclass: Do not use -pie with hardening
Fix build when PIE is turned on. It tries to build
.so file using -pie and -shared flags together because
its doing compile and link in same step CFLAGS and LDFLAGS
are combined and does not work, ending in errors e.g.
| /mnt/a/oe/build/tmp/work/cortexa7t2hf-neon-vfpv4-bec-linux-musleabi/python-pygpgme/0.3-r0/recipe-sysroot/usr/l
ib/Scrt1.o: In function `_start_c':
| /usr/src/debug/musl/1.1.16+gitAUTOINC+179766aa2e-r0/git/crt/crt1.c:17: undefined reference to `main'
| collect2: error: ld returned 1 exit status
This error while cryptic is due to the fact that we are
building a shared library but also pass -pie flag to the link
step after specify LDHSARED ( which is -shared linker flags )
we can not use -pie when doing shared libs. This is true for all the python
modules inheriting setup tools
Disable the pie flags thusly for all modules using setuptools since
this setting is done in setuptools makefiles which are then used
during module compiles
Signed-off-by: Khem Raj <raj.khem@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
David Vincent [Fri, 2 Jun 2017 15:16:03 +0000 (17:16 +0200)]
initscripts: Populate volatile from existing file
In some cases, it may be useful to populate a volatile file from an
existing one, e.g. a file in a read-only rootfs that may be edited in a
read-write destination.
To provide this behavior, creation of volatile files has been updated to
copy a file which has been given in the <linksource> field. If set to
none, the current behavior is preserved.
Signed-off-by: David Vincent <freesilicon@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Tue, 30 May 2017 08:08:01 +0000 (10:08 +0200)]
sstate-diff-machines.sh: Replace MACHINE_ARCH only at the beginning and separated with dash
* I had some components where the MACHINE name was also included in PV of allarch recipe
which was tripping the script into saying that they have different signatures (because
for one MACHINE was the string in PV replaced with 'MACHINE' and not for other MACHINEs
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Martin Jansa [Tue, 30 May 2017 08:08:36 +0000 (10:08 +0200)]
recipes: Replace "cp -a" with "cp -R --no-dereference --preserve=mode, links"
* Using "cp -a" leaks UID of user running the builds, causing
many QA warnings.
* See this thread for details:
http://lists.openembedded.org/pipermail/openembedded-core/2015-November/112904.html
Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
scripts/oe-selftest: Migrate to new framework into oeqa.selftest.context
The new OEQA framework aims to re-use code into the different Test
components.
The previous oe-selftest implements it-self loading, run, and list test
cases in a non-standard way (unittest base) and other functionalities
like logging that is now on oeqa core. This ends on a compact oe-selftest
script.
All needed command line options was migrated but there are some of them
pending of implementation and others deprecated.
Deprecated options:
list-tags: The tag functionality into the old oeqa framework isn't
work, the selftest doesn't has tag decorators.
{run, list}-tests-by: Ambiguos options it accepts all the posibilites module,
class, name, id or tag.
Remaining to implement:
coverage: It enables covrage reports over a test run, currently isn't on
on use and some bugs [1], i filed a bug to add support to OEQA core module in
this way other Test components could enable it.
repository: It push XML results into a git repository and isn't in use,
i filed a bug to implement this into OEQA core module. [2]
scripts/oe-selftest: Move {add,remove}_include files to case
The oe-selftest creates include files to store custom configuration to
make specific tests, every class executes a different test and may be
uses custom configuration.
So move to case class in order to simplify oe-selftest script and later
implement later a build folder per class.
oeqa/selftest/case: Migrate case class to the new OEQA framework
Summary of the changes:
- Use OETestCase as base class instead of unittest.TestCase
- Remove LogResults decorator the new framework provides logging into
the core functionality.
- Logger is now self.logger instead of self.log
- Move comments into docstrings in several help methods
- Use get_test_layer() method instead of access monkey patched variable
in old oeSelfTest case class.
Aníbal Limón [Tue, 23 May 2017 20:04:57 +0000 (15:04 -0500)]
scripts/oe-test: Move load_test_components to oeqa.utils
In order to maintain compatibility with oe-selftest, the
load_test_components needs to be re-used, so the script
executor needs to pass to only load components supported
by certain script (oe-test, oe-selftest).
Ross Burton [Mon, 5 Jun 2017 16:30:42 +0000 (17:30 +0100)]
oeqa/selftest: lock down Meson git revision for reliability
The test_recipetool_create_github test fetches HEAD of the repository so
upstream changes can (and do) break the test. Avoid these problems by passing
the rev= argument in the URL to lock the checkout to the same version that is
fetched in the github_tarball test.
Also pass the commands to runCmd() as a list instead of a string, the semicolon
in the URL needs more quotes if the shell is involved and passing a list
bypasses the shell entirely.
Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
kernel.bbclass: fix KERNEL_IMAGETYPE(S) for Image.gz
KERNEL_IMAGETYPES lists all the kernel images that we want to build. in cb17b6c2a7 (kernel.bbclass: support kernel image type of vmlinux.gz), some logic
was added to support vmlinux.gz which is not a target built by kernel
makefiles (only vmlinux). It is clear that the goal of this logic is only to
support vmlinux.gz and not others compressed format (such as Image.gz) which are
valid target for kernel makefiles.
For Image.gz we should rely on the kernel makefiles and not do the compression
in kernel class.
This patch updates the logic used to filter out non supported kernel target from
KERNEL_IMAGETYPES, and make vmlinux.gz a 'special case', instead of *.gz. If
more special cases are needed in the future, we could add them in a similar way.
This patch should be a no-op for anyone using vmlinux or vmlinux.gz, and on top
of that it is fixing the build for Image.gz which was not working until now.
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Jussi Kukkonen [Thu, 1 Jun 2017 10:36:46 +0000 (13:36 +0300)]
vte: Upgrade 0.46.1 -> 0.48.3
Depend on gperf-native: The tarball used to include files generated
with gperf and this is no longer the case.
Use GIR_EXTRA_LIBS_PATH to fix introspection generation.
Signed-off-by: Jussi Kukkonen <jussi.kukkonen@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
* Costmetic changes in license.html file required new MD5 hash
* Add PACKAGECONFIG option for libunwind support (new feature, disable by default)
* Rebased 0001-Use-wayland-scanner-in-the-path.patch
* Added zlib in DEPENDS, otherwise mesa no longer builds.
* Added 0001-util-rand_xor-add-missing-include-statements.patch, sent upstream
Signed-off-by: Nicolas Dechesne <nicolas.dechesne@linaro.org> Signed-off-by: Ross Burton <ross.burton@intel.com>
Maxin B. John [Wed, 31 May 2017 11:39:35 +0000 (14:39 +0300)]
libtasn1: upgrade 4.10 -> 4.12
Noteworthy changes:
1. Introduced the ASN1_TIME_ENCODING_ERROR error code to indicate an invalid
encoding in the DER time fields.
2. Introduced flag ASN1_DECODE_FLAG_ALLOW_INCORRECT_TIME. This flag allows
decoding errors in time fields even when in strict DER mode.
3. Added safety check in asn1_find_node(). That prevents a crash when a very
long variable name is provided by the developer.
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Maxin B. John [Wed, 31 May 2017 11:39:34 +0000 (14:39 +0300)]
sqlite3: upgrade 3.18.0 -> 3.19.2
Bug fixes:
1. Fix a problem in REPLACE that can result in a corrupt database
containing two or more rows with the same rowid.
2. Fix a problem in PRAGMA integrity_check that was causing a subsequent
VACUUM to behave suboptimally.
3. Fix the PRAGMA foreign_key_check command so that it works correctly with
foreign keys on WITHOUT ROWID tables.
4. Disallow leading zeros in numeric constants in JSON.
5. Disallow control characters inside of strings in JSON.
6. Limit the depth of recursion for JSON objects and arrays in order to
avoid excess stack usage in the recursive descent parser.
7. Fix more bugs in the LEFT JOIN flattening optimization.
Signed-off-by: Maxin B. John <maxin.john@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
start_getty: Over added SERIAL_CONSOLE cause error in userspace log
Error log will be logged into /var/log/message.
Added in more condition checking on the script. Check
/proc/tty/drivers and /proc/tty/driver/*
file system to retrieve active targeted serial.
Only establish getty with active serial in runtime.
Mikko Rapeli [Thu, 1 Jun 2017 15:52:59 +0000 (18:52 +0300)]
scripts: Fix return value checks from subprocess.call()'s
Python function subprocess.call() returns the return value of the
executed process. If return values are not checked, errors may
go unnoticed and bad things can happen.
Change all callers of subprocess.call() which do not check for
the return value to use subprocess.check_call() which raises
CalledProcessError if the subprocess returns with non-zero value.
Ed Bartosh [Wed, 17 May 2017 12:49:12 +0000 (12:49 +0000)]
isoimage-isohybrid: don't use TRANSLATED_TARGET_ARCH
isoimage-isohybrid plugin fails with this error when
trying to find initrd image:
initrd = glob.glob('%s/%s*%s.%s' % (initrd_dir, image_name, target_arch, image_type))[0]
IndexError: list index out of range
as it uses TRANSLATED_TARGET_ARCH value as part of the image.
This approach stopped to work due to changes in oe core code.
initrd file name is made using MACHINE variable. wic can't get value
of this variable as it's not included into bitbake -e output.
Used basename of deploy dir as MACHINE value to fix the breakage.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
insane.bbclass: fix override handling in RDEPENDS QA
The package_qa_check_rdepends() in insane.bbclass has
incorrectly replace its localdata OVERRIDES value with
the package name. Fixing it by appending the package name
to the existing OVERRIDES value. This resolves RDEPENDS QA
error when setting PACKAGECONFIG using a pn- override at
local.conf.
[YOCTO #11374]
Signed-off-by: Gan, Yau Wai <yau.wai.gan@intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Daniel Lublin [Tue, 30 May 2017 11:17:59 +0000 (13:17 +0200)]
devtool: deploy-target: Let script continue even if rm fails
The remote script is run with `set -e`, so doing rm without -f on a
$file that is already gone will exit the whole script, failing any
redeployment. Assume a use case where packages sometimes produces
certain test binaries stored on volatile media (tmpfs), and where the
system is occasionally rebooted.
Signed-off-by: Daniel Lublin <daniel@lublin.se> Signed-off-by: Ross Burton <ross.burton@intel.com>
Ola x Nilsson [Tue, 30 May 2017 10:16:31 +0000 (12:16 +0200)]
devtool: Compare abspath of both B and S
Either both or none of the paths must be passed through
os.path.abspath or things like 'A//B', 'A/./B/', and 'A/B/' in S will
cause unintentional mismatches even when B = "${S}".
Using os.path.abspath for both seems more likely to be correct as that
will also handle the case where ${B} != ${S} but the abspaths are
equal.
Signed-off-by: Ola x Nilsson <olani@axis.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 1 Jun 2017 01:42:25 +0000 (13:42 +1200)]
classes/buildhistory: make a single commit per build
Way back in OE-Core commit fba198ac7efe476a25c5761878ef2fcee97bf9f1 in
2012 we split committing to the repository, making a commit per
top-level directory. However, as we add more information it becomes
harder to see which commits belong to which build. Switch back to a
single commit per build to keep the history tidier.
To address the original concern, if you do want to see just the changes
for a particular subdirectory, git can filter that for you - just
specify that subdirectory as the last parameter on the git show / git
diff command line and that's all you will see.
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
oeqa/core/context: Omit docstring output on XMLResult
By default, the xml runner class prints out the docstring for every
unit test but it order to keep the same format as the standard
runner, avoid docstring output setting descriptions to False.
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
oeqa/core/context: Use a default iterable modules object
The OETestLoader.modules must be an iterable object, otherwise
checking presence of specific modules with the 'in' operator fails
with the following command/error:
% oe-test core
.. - core - INFO - Running bitbake -p
Traceback (most recent call last):
File "..poky2/scripts/oe-test", line 108, in <module>
ret = main()
File "..poky2/scripts/oe-test", line 93, in main
results = args.func(logger, args)
File "..poky2/meta/lib/oeqa/core/context.py", line 235, in run
self.tc.loadTests(self.module_paths, **self.tc_kwargs['load'])
File "..poky2/meta/lib/oeqa/core/context.py", line 53, in loadTests
self.suites = self.loader.discover()
File "..poky2/meta/lib/oeqa/core/loader.py", line 204, in discover
pattern='*.py', top_level_dir=path)
File "/usr/lib64/python3.4/unittest/loader.py", line 275, in discover
tests = list(self._find_tests(start_dir, pattern))
File "/usr/lib64/python3.4/unittest/loader.py", line 339, in _find_tests
yield self.loadTestsFromModule(module)
File "..poky2/meta/lib/oeqa/core/loader.py", line 266, in loadTestsFromModule
and module.__name__ in self.modules \
TypeError: argument of type 'NoneType' is not iterable
Signed-off-by: Leonardo Sandoval <leonardo.sandoval.gonzalez@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:43 +0000 (15:37 -0500)]
oeqa/sdk: Enable usage of OEQA thread mode
oeqa/sdk/context.py - Use OETestContextThreaded.
classes/testsdk.bbclass - Enable bb event thread mode to avoid
corrupt the PIPE when multiple threads writes.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:41 +0000 (15:37 -0500)]
oeqa/core/tests: Add tests of OEQA Threaded mode
Add needed tests to validate the OEQA Threaded mode, the remaining
parts are tested by the OEQA without Threaded mode.
- test_loader.py: Add a test to validate rules when creating the
list of test suites.
- test_decorators.py: Add oetimeout test because the threaded mode
uses Timer instead of signal.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:40 +0000 (15:37 -0500)]
oeqa/core/decorator/oetimeout: Add support for OEQA threaded mode
In python signals are only allowed to delivery into the main thread,
to support the threading mode test if the runner is threaded and
use threading.Timer instead.
There are some considerations like SIGALRM interrupts the execution
after N seconds but the Timer only starts a Thread to notice the
timeout and the exception will be raised when the test run ends.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
The OETestResultThreadedInternal extends OETestResult to stores
results by Thread.
The OETestResultThreaded is a simple class that provides the
implementation of interfaces needed by outside like wasSuccesful,
stop, logSummary, logDetails.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:34 +0000 (15:37 -0500)]
oeqa/core/threaded: Add OEStreamLoggerThreaded class
The OEStreamLoggerThreaded overrides OEStreamLogger to redirect
the PyUnit output to a logger.
Instead of log every line when comes the OEStreamLoggerThreaded
will buffer the PyUnit output and write everything at end of every
suite execution to don't have mixed suite outputs.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:33 +0000 (15:37 -0500)]
oeqa/core/threaded: Add new module with OETestLoaderThreaded
The threded module will implement Thread support into the OEQA
framework.
The OETestLoaderThreaded overrides discover to return a list of
suites every suite will be run by a separate Thread.
Some design considerations are:
- All the tests of a certain module needs to be run at one thread
because unittest framework needs to keep track of setUp{Module,
Class,} and tearDown{Module, Class,}.
- Tests that depends on other needs to be run at the same thread
because OEQA framework look at the status of dependant test to
decide if skip or not, this constraint can be change in the future
but a sync mechanishm is needed to implement between the Threads.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
Aníbal Limón [Fri, 26 May 2017 20:37:30 +0000 (15:37 -0500)]
oeqa/core: Don't expose OEStreamLogger in OETestContext
The OEStreamLogger class is used for redirect PyUnit output
to a certain logger so there is not need to expose at level
of OETestContext because only OETestRunner needs to know.
[YOCTO #11450]
Signed-off-by: Aníbal Limón <anibal.limon@linux.intel.com> Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
