selftest: automatically add the meta-selftest layer
adding the meta-selftest layer is a small bump you always encounter
when you want to run selftest. Adding an automatic procedure to include
the layer if it is not yet present.
Signed-off-by: Benjamin Esquivel <benjamin.esquivel@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Paul Eggleton [Thu, 19 Jan 2017 20:36:00 +0000 (09:36 +1300)]
devtool: check locale and refuse to start if it isn't UTF-8
We need to ensure the locale is UTF-8 or otherwise strange errors will
occur later on during execution - the same reason we check this in
BitBake itself. Unfortunately this check has to be before command line
parsing and therefore showing the help text in response to --help, since
that relies upon parsing bitbake's configuration (as we need to load
plugins in other layers).
Fixes [YOCTO #10908].
Signed-off-by: Paul Eggleton <paul.eggleton@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Robert Yang [Tue, 29 Nov 2016 06:02:05 +0000 (22:02 -0800)]
runqemu: fixes for slirp, network device and hostfwd
Fixed:
- Add QB_NETWORK_DEVICE to set network device, it will be used by both
slirp and tap.
- Set QB_NETWORK_DEVICE to "-device virtio-net-pci" in qemuboot.bbclass
but runqemu will default to "-device e1000" when QB_NETWORK_DEVICE is
not set, this is because oe-core's qemu targets support
virtio-net-pci, but the one outside of oe-core may not,
"-device e1000" is more common.
- Set hostfwd by default: 2222 -> 22, 2323 -> 23, and it will choose a
usable port when the one like 222 is being used. This can avoid
conflicts when multilib slirp qemus are running. We can forward more
ports by default if needed, and bsp.conf can custom it.
- Use different mac sections for slirp and tap to fix conflicts when
running both of them on the same host.
[YOCTO #7887]
CC: Nathan Rossi <nathan@nathanrossi.com> CC: Randy Witt <randy.e.witt@linux.intel.com> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Robert Yang [Wed, 23 Nov 2016 08:57:39 +0000 (00:57 -0800)]
runqemu: support multiple qemus running when nfs
Fixed:
* In build1:
$ runqemu nfs qemux86-64
In build2:
$ runqemu nfs qemux86-64
It would fail before since the port numerbs and conf files are
conflicted, now make runqemu-export-rootfs work together with runqemu to
fix the problem.
* And we don't need export PSEUDO_LOCALSTATEDIR in runqemu, the
runqemu-export-rootfs can handle it well based on NFS_EXPORT_DIR.
* Remove "async" option from unfsd to fix warning in syslog:
Warning: unknown exports option `async' ignored
* Fixed typos
Both slirp and tap can work.
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
Chen Qi [Fri, 23 Dec 2016 02:59:21 +0000 (10:59 +0800)]
package_manager: default to have scriptlet output captured in log
We need to have scriptlet output captured in log. If we don't do so,
some useful information from scriptlets (especially postinstall script)
would be missing. In case a script has a warning message but it does not
necessarily have to fail, the message should be captured.
Opkg has already done that. Change for rpm and dpkg so that scriptlet
output is captured and no warning message is missing.
Signed-off-by: Chen Qi <Qi.Chen@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Markus Lehtonen [Thu, 19 Jan 2017 10:16:45 +0000 (12:16 +0200)]
oeqa.buildperf: store measurements as a dict (object) in the JSON report
Store measurements as a dict, instead of an array, in the JSON report.
This change makes traversing of the report much easier. The change also
disallows identically named measurements under one test, as a sanity
check for the test cases.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Markus Lehtonen [Wed, 28 Dec 2016 14:01:21 +0000 (16:01 +0200)]
oe-build-perf-test: save test metadata in a separate file
The patch introduces a new metadata (.json or .xml) file in the output
directory. All test meta data, e.g. git revision information and tester
host information is now stored there. The JSON report format is slightly
changed as the metadata is not present in results.json anymore.
[YOCTO #10590]
Signed-off-by: Markus Lehtonen <markus.lehtonen@linux.intel.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Bruce Ashfield [Fri, 13 Jan 2017 20:40:38 +0000 (15:40 -0500)]
kern-tools: scc: Fix process_file error check
Commit 626ceac135fa66277c2fa53197be33cc9d4d7614 broke the error
check in process_file by adding in three lines that stomp on $? which
print the output file when verbose is set.
Move output file on verbose print to an elif after the error check.
Signed-off-by: George McCollister <george.mccollister@gmail.com> Signed-off-by: Bruce Ashfield <bruce.ashfield@windriver.com> Signed-off-by: Ross Burton <ross.burton@intel.com>
Tanu Kaskinen [Fri, 20 Jan 2017 09:26:28 +0000 (11:26 +0200)]
alsa-tools: 1.1.0 -> 1.1.3
Drop 0001-Cus428Midi-Explicitly-cast-constant-to-char-type.patch,
because the new release has an equivalent fix (and that's actually the
only change in the new release).
Signed-off-by: Tanu Kaskinen <tanuk@iki.fi> Signed-off-by: Ross Burton <ross.burton@intel.com>
Mariano Lopez [Wed, 18 Jan 2017 13:19:52 +0000 (13:19 +0000)]
oeqa/runtime/context.py: Add defaults for runtime context
This adds default values to OERuntimeTestContextExecutor class in
order to make easier the execution of exported test that were
generated with testexport class.
Mariano Lopez [Mon, 9 Jan 2017 14:49:57 +0000 (14:49 +0000)]
testimage.bbclass: Add support for package extraction
testimage support the installation of packages without a package
manager in the target. This adds support for package extraction
required to support the installation feature.
Mariano Lopez [Tue, 3 Jan 2017 08:18:00 +0000 (08:18 +0000)]
runtime/cases/smart.py: Migrate smart tests
This migrates the smart test from the old framework to
the new one. This has its own commit because smart
test was using bb and oe libraries that are available
when exporting the test cases to run in a different host.
Because of the removal of bb and oe libraries index and
packages feeds creation will be managed in testimage bbclass.
Mariano Lopez [Wed, 7 Dec 2016 13:35:43 +0000 (13:35 +0000)]
oeqa/runtime/files: Move runtime files from old directory
As part of the refactor we require to move the files used
in runtime testing to the new directory. This also adds
the path to the runtime test context.
Mariano Lopez [Mon, 31 Oct 2016 13:15:02 +0000 (13:15 +0000)]
oeqa/runtime: Add case, context and loader classes for runtime testing
This adds OERuntimeTestCase, OERuntimeTestContext, and OERuntimeTestLoader
to be used for runtime testing.
As expected there are some changes in runtime context:
- Adds the target to be used for runtime testing, the default
is a SSH connection to the device under test running a OE image.
- Runtime context requires image manifest because several
tests are skipped if a package is missing or installed.
- Several tests require the output of the ps command and it changes
its output and arguments if busybox o procps is installed, so the
case must use the correct ps command.
Aníbal Limón [Thu, 27 Oct 2016 21:39:47 +0000 (16:39 -0500)]
oeqa/runtime: Move to runtime_cases
The new oeqa core framework will modify the structure of the runtime
folder the new runtime folder will have python code inside to support
runtime test cases.
Aníbal Limón [Wed, 30 Nov 2016 21:12:15 +0000 (15:12 -0600)]
oeqa/sdkext/cases: Migrate test case to new OEQA framework
Summary,
- Changes base case class to OESDKExtTest.
- Changes decorator classes to new ones.
- Chnages variable names sdktestdir -> sdk_dir.
- Added missing license to MIT.
Aníbal Limón [Wed, 30 Nov 2016 00:12:49 +0000 (18:12 -0600)]
classes/testsdk: Remove the need of TEST_LOG_DIR variable
The TEST_LOG_DIR was used for store sdk_target_log this log
contains the output of the run of build commands now that information
could be found also on log.do_testsdk under WORKDIR.
The log will continue to store into SDK_DIR instead of TEST_LOG_DIR.
Aníbal Limón [Sun, 27 Nov 2016 23:51:53 +0000 (17:51 -0600)]
oeqa/sdk/cases: Migrate tests to the new OEQA framework
Summary of the changes:
- Remove auto extend_path using pkgutil at __init__, is not needed.
- Change base class to OESDKTestCase.
- Add td_vars attr to set dependencies of certain variables in test
data.
- Change skips from module level to class level because Test context
(tc)
now isn't at module level.
- Variable names changes to be consistent (i.e. sdktestdir ->
sdk_dir).
[YOCTO #10599]
- Don't use bb.utils functions use instead remove_safe and shutil
for copy files.
- SDKBuildProject pass test data variables instead of call getVar
inside.
Aníbal Limón [Wed, 2 Nov 2016 19:42:01 +0000 (13:42 -0600)]
oeqa/utils: {Target,SDK,}BuildProject remove dependency of bb
Don't use bitbake references inside utils modules, in order todo
that changes getVar calls for arguments in the __init__ method like
dl_dir for all the classes and testlogdir, builddatetime in
SDKBUildProject.
Also don't export proxies inside _download_archive method, a good
practice is to setup the proxies at init of the process instead of
do it in this helper module.
Aníbal Limón [Mon, 31 Oct 2016 23:24:25 +0000 (17:24 -0600)]
oeqa/sdk: Add case and context modules for the SDK component
Adds case and context modules for SDK based on oetest.py old code.
Enables SDK Test component usage with oe-test, the SDK Test component
adds command line options for specify sdk installed dir, sdk environment
and target/hosts maniftest.
Mariano Lopez [Wed, 21 Dec 2016 13:14:00 +0000 (13:14 +0000)]
oeqa/utils/__init__.py: Adds compatibility with bitbake logger
The bitbake logger changes the way debug is logged and adds
different levels within debug, this is passed as argument
to the function and breaks compatibility with vanilla loggers.
This implements a way to handle this adding a new function for
debug, that will dispatch the correct logging method signature.
Also overrides info method to use logging.INFO + 1 in order to
see plain data.
Also this commit fix the issue of not showing the test summary
and results when running from bitbake.
Aníbal Limón [Fri, 25 Nov 2016 17:36:06 +0000 (11:36 -0600)]
oe/data: Add export2json function
The export2json function export the variables contained in
the data store to JSON format, the main usage for now will be
to provide test data to QA framework.
Aníbal Limón [Wed, 9 Nov 2016 17:50:46 +0000 (11:50 -0600)]
oeqa/core/context: Add support of OETestContextExecutor
The OETestContextExecutor class supports to use oe-test for run core
test component also is a base class for the other test components
(runtime, sdk, selftest).
Te principal functionality is to support cmdline parsing and execution
of OETestContext, the test components could extend the common options
to provide specific ones. The common options between test components
are test data file, output log and test cases path's to scan.
Also it initializes the logger to be passed to the whole OEQA framework.
Aníbal Limón [Mon, 31 Oct 2016 20:42:30 +0000 (14:42 -0600)]
scripts/oe-test: Add new oe-test script
The new oe-test script will be use to run test components with
one single script.
The oe-test script search for test components inside meta/lib/oeqa,
the test components needs to implement OETestContextExecutor inside
context module in order to be supported by oe-test.
Mariano Lopez [Wed, 9 Nov 2016 17:19:07 +0000 (11:19 -0600)]
oeqa/core/decorator: Add support for OETimeout decorator
The OETimeout provides support for specify certain timeout
in seconds for a test case, if the timeout is reach the SIGALRM
is sent and an exception is raised to notify the timeout.
Aníbal Limón [Wed, 9 Nov 2016 17:09:07 +0000 (11:09 -0600)]
oeqa/core/decorator: Add support for OETestDepends
The OETestDepends decorator could be used over test cases to
define some dependency between them.
At loading time sorting the tests to grauntee that a test case
executes before also raise an exception if found a circular
dependency between test cases.
At before test case run reviews if the dependency if meet, in the
case of don't it skips the test case run.
Aníbal Limón [Wed, 9 Nov 2016 16:38:37 +0000 (10:38 -0600)]
oeqa/core: Add loader, context and decorator modules
loader: Implements OETestLoader handling OETestDecorator
and filtering support when load tests. The OETestLoader is
responsible to set custom methods, attrs of the OEQA
frameowork.
[YOCTO #10231]
[YOCTO #10317]
[YOCTO #10353]
decorator: Add base class OETestDecorator to provide a common
way to define decorators to be used over OETestCase's, every
decorator has a method to be called when loading tests and
before test execution starts. Special decorators could be
implemented for filter tests on loading phase.
context: Provides HIGH level API for loadTests and runTests
of certain test component (i.e. runtime, sdk, selftest).
Mariano Lopez [Wed, 9 Nov 2016 16:33:42 +0000 (10:33 -0600)]
oeqa/core: Add utils module for OEQA framework
misc: Functions for transform object to other types.
path: Functions for path handling.
test: Functions for operations related to test cases and suites.
Aníbal Limón [Tue, 8 Nov 2016 23:57:43 +0000 (17:57 -0600)]
oeqa/core: Add base OEQA framework
case: Defines OETestCase base class that provides custom
methods/attrs defined by the framework.
Every OETestCase instance contains a reference to the test
data (d), the test context (tc) and the logger.
Also implements _oe{SetUp,TearDown}Class for make special
handling of OEQA decorators and validations.
runner: Defines OETestRunner/OETestResult with support for RAW
and XML result logs.
exception: Custom exceptions related to the OEQA framework based
on class OEQAException.
Ed Bartosh [Wed, 18 Jan 2017 17:24:27 +0000 (19:24 +0200)]
selftest: wic: don't set WKS_FILE
Setting WKS_FILE variable should be done only when
wic image is expected to be built by bitbake.
If it's set for all images it breaks image building in
some cases.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Wed, 18 Jan 2017 17:21:17 +0000 (19:21 +0200)]
wic: fixed test_iso_image test case
Fixed isoimage-isohybrid plulgin and correspondent wic tet case:
- used wic-tools target when getting varlue of STAGING_LIBDIR variable
- ensured that image is built with efi and hddimg enabled
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Mon, 16 Jan 2017 20:53:46 +0000 (22:53 +0200)]
image.bbclass: put .env files to ${STAGING_DIR}/imgdata/
As STAGING_DIR_TARGET started to point to a recipe specific
sysroot wic is not able to add .env files when .wks file refers
to multiple rootfs recipes.
Used STAGING_DIR instead of STAGING_DIR_TARGET to make the
directory with .env files the same for all recipes.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Ed Bartosh [Mon, 16 Jan 2017 20:42:29 +0000 (22:42 +0200)]
wic: fix getting path of native sysroot
wic used STAGING_DIR_NATIVE variable as a path to native sysroot.
This doesn't work with recipe specific sysroots as STAGING_DIR_NATIVE
points to the native sysroot of the current recipe.
Used RECIPE_SYSROOT_NATIVE variable of wic-tools recipe
to fix the issue.
Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
