]> code.ossystems Code Review - openembedded-core.git/commit
oeqa/logparser: Fix performance issues with ptest log parsing
authorRichard Purdie <richard.purdie@linuxfoundation.org>
Thu, 20 Feb 2020 13:57:50 +0000 (13:57 +0000)
committerRichard Purdie <richard.purdie@linuxfoundation.org>
Fri, 21 Feb 2020 09:37:04 +0000 (09:37 +0000)
commit01c9f40b979e36a53e789a6bedd89b6d9557dce3
treea2be2e0842b92c420eac443b50c0da8cc84dd4d2
parentb266a1ac662405e5bab98657c30b1a13cd86d046
oeqa/logparser: Fix performance issues with ptest log parsing

On the autobuilder a ptest log with 2.1 million lines took around 18 hours to
process. This is clearly crazy.

We can tweak the processing code to:
a) Stop repeatedly joining large strings together (append to a list instead)
b) Use one startswith expression instead of multiple re.search() operations

With this change it takes 18 hours down to around 12s.

[YOCTO #13696]

Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>
meta/lib/oeqa/utils/logparser.py