lib/oe/utils: add directory size function

For the purpose of image construction using du on a rootfs directory isn't
entirely satisfactory.  Bare "du" will report the actual disk usage so file
systems which can compress the data will report less than the actual space
required.  Using "du --apparent-size" will report the actual space used, but as
this simply sums the bytes used for content across an entire file system can
result in significant under-reporting due to block size overhead.

Attempt to solve these problems by implementing our own function to calculate
how large a rootfs will be.  This function handles hardlinks correctly but
rounds up all sizes to multiples of the block size (currently, 4KB is the
hard-coded block size).

Signed-off-by: Ross Burton <ross.burton@intel.com>
Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org>

diff --git a/meta/lib/oe/utils.py b/meta/lib/oe/utils.py
index 9a2187e36f8156cf96fa3b8ca793e1a81acc0188..a84039f58552c208a2aab8fc41b9e1b612058944 100644 (file)
--- a/meta/lib/oe/utils.py
+++ b/meta/lib/oe/utils.py
@@ -536,3 +536,34 @@ class ImageQAFailed(Exception):
 def sh_quote(string):
     import shlex
     return shlex.quote(string)
+
+def directory_size(root, blocksize=4096):
+    """
+    Calculate the size of the directory, taking into account hard links,
+    rounding up every size to multiples of the blocksize.
+    """
+    def roundup(size):
+        """
+        Round the size up to the nearest multiple of the block size.
+        """
+        import math
+        return math.ceil(size / blocksize) * blocksize
+
+    def getsize(filename):
+        """
+        Get the size of the filename, not following symlinks, taking into
+        account hard links.
+        """
+        stat = os.lstat(filename)
+        if stat.st_ino not in inodes:
+            inodes.add(stat.st_ino)
+            return stat.st_size
+        else:
+            return 0
+
+    inodes = set()
+    total = 0
+    for root, dirs, files in os.walk(root):
+        total += sum(roundup(getsize(os.path.join(root, name))) for name in files)
+        total += roundup(getsize(root))
+    return total