Rouge User Reference Manual

About

rouge is a companion tool for moulin. Its purpose is to simplify the creation of bootable images. It can create a partition table, fill partitions with predefined files or raw data. It supports GPT, ext4fs, raw images, and Android sparse images. Further formats can be added if needed.

Right now, it can be used only as a separate tool, but there are plans to integrate it into moulin output.

Design Principles

rouge shares ideas (and code) with moulin. Thus, it is very similar in configuring and invoking to moulin. It can be used as a stand-alone tool: provide only images: section in your YAML file. Or you can include this section in the same file, which is used by moulin to share common options or variables. In the latter case, moulin will generate additional image-{image_name} rules so that you can build images with Ninja.

Requirements

To perform its function, rouge invokes several external utilities. Most of them are available on every system, with the following exceptions:

  • To work with vfat, install the mtools package.

  • To work with Android, install the simg2img package.

There is a list of external tools used for the generation of images:

  • dd - used to copy raw images

  • mkfs.ext4 - creates ext4 FS

  • mkfs.vfat - creates vfat FS

  • simg2img - used to unpack Android sparse image files

  • mcopy and mdd - to handle vfat images

Invoking rouge

rouge uses the same design ideas as moulin, and part of the command line options are shared with moulin. This includes --help-config, -v, and --dump arguments. Please refer to moulin documentation for more details. This document describes only arguments specific to rouge.

rouge [-h] [--help-config] [-v] [--dump] [-l] [-f] [-s] [-o FILE]
             [-l | -i image_name]
             build.yaml image_name

  • -i image_name - name of one of the images described in images: section of your build configuration file (build.yaml). Basically, this is the image you want to create.

  • -l, --list-images - list available images and their descriptions. Please note that the actual list of images can depend on build config parameter values. For example, your build config may provide an option to enable Android build. If this option is enabled, you may have a separate image for Android.

  • -f, --force - force overwrite existing file. If this option is not given, rouge will refuse to create an image if the output file already exists.

  • -s, --special - allow to write to a special file, like a block device. Without this option, rouge will refuse to write to, say, /dev/sda. Use this option with care and always double-check the device name, as rouge will overwrite anything that is stored on that device.

  • -o - provides output file name. This is an optional parameter, by default, rouge will write to <image_name>.img.

Apart from these options, rouge will read and parse all YAML file-related parameters in the same way as moulin does. You can check available parameters with --help-config.

Principles of Operation

rouge works in a very simple way. It uses moulin’s YAML processor that applies parameters and substitutes variables, then reads images: section, finds the requested image specification.

For a given image, it checks if all mentioned files are present, then calculates the sizes of partitions (if any) and the total image size. Then it writes data to a given file/block device according to the specifications.

rouge tries to use sparse files whenever possible. A sparse file is a file with “holes” in it. It allows you to have a huge file that represents a whole disk image with a tiny bit of actual information in it. This speeds up the image creation process and decreases the used disk space. If you are writing the resulting image file to your SD card manually, try adding conv=sparse option to your dd command line. This will speed up the writing process. If you want to distribute resulting images, take a look at Intel’s bmap tool. It allows you to share sparse files across devices.

YAML Sections

Shared sections

rouge uses the same YAML processing code as moulin so refer to moulin’s documentation for the desc, min_ver, variables, parameters sections description. This page describes only parts specific to rouge.

Image specifications

Images are specified in the following way:

images:
  image_name_a:
    desc: "Description for the first image"
    image_size: 512 MiB
    type: gpt
    ... block description ...
  image_name_b:
    desc: "Description for the second image"
    type: raw_image
    ... block description ...
  image_name_c:
    desc: "Description for the third image"
    type: empty
    ... block description ...

images: section contains one or more keys, which serve as image names. Every image can have a description, which will be displayed when rouge lists available images. type: key is mandatory as it defines the type of block. Supported block types are described in the following sections.

Also, you may specify the required size of the image using image_size:. Please see the section ‘Size Designation’ below for supported notation. If the actual size of all partitions will be less than image_size: then image will be blown up to image_size:. If the actual size is bigger than specified, an error will be printed with an explanation like “Actual size (20000) of image is bigger than requested one (10000).”

Block descriptions

“Block” is a basic rouge entity that describes one partition or partition table. Some block types can be nested. Supported block types are described below.

Size Designation

All blocks have size parameter. For some block types, this parameter is mandatory, for some - optional. The basic unit for size is a byte. For example

type: empty
size: 4096

defines an empty block with a size of 4096 bytes. rouge supports some SI suffixes:

  • KB - kilobyte - 1000 bytes

  • MB - megabyte - 1000 kilobytes or 1 000 000 bytes

  • GB - gigabyte - 1000 megabytes or 1 000 000 000 bytes

  • KiB - kibibyte - 1024 bytes

  • MiB - mebibyte - 1024 kibibytes or 1 048 576 bytes

  • GiB - gibibyte - 1024 mebibytes or 1 073 741 824 bytes

The suffix must be separated from the number by a space. For example: size: 4 MiB defines the size of 4 mebibytes or 4 194 304 bytes.

On the sparse option

Almost all block descriptions support boolean sparse option, which is enabled by default. You can disable it to generate non-sparse parts of the resulting images. This will create images that are bigger while stored on disk, because they physically store all non-needed NUL regions. But this may be used in cases when you need to zero out some regions on flash storage. Bear in mind that in this case, you can’t write the result image with

dd of=image.img of=/dev/outdevce conv=sparse

because with conv=sparse option dd will “un-sparse” the image file, effectively skipping big zeroed regions. So, you either need to remove conv=sparse option when calling dd, increasing writing time significantly, or use bmaptool which should be less aggressive with sparsed regions detection.

Empty block

An empty block is a block that does not contain any file or raw image. rouge will write nothing into this block if filled: zeroes option is not specified.

type: empty # defines empty block
size: 4096
filled: zeroes

size is mandatory, as rouge can’t infer it.

filled is optional, with only zeroes value allowed for now. This option may be used if you need the block to be filled with zeroes. For example, this is used for some Android partitions, like ‘rpmbemul’. You can use this option only if absolutely necessary. Otherwise, you will needlessly increase the size and upload time of an image.

Raw Image Block

The purpose of this block type is to include any binary data from another file. For example, if your build system creates a .ext4 image with the root file system, you can use this block to place that image into a GPT partition (which is described below).

type: raw_image # defines raw image block
size: 400 MiB
resize: false
image_path: "some/path/rootfs.ext4"

image_path is mandatory. This is a file to be included in resulting image.

size is optional. If it is omitted, rouge will use the size of the file. If provided size is smaller than file size, rouge will stop with an error. If provided size is bigger than file size, rouge will try to resize the file to match size. This rule applies to ext2..ext4 format now. Note that host tools perform resizing, and you may meet some compatibility issues if the newer tools generate the ext4 image. For example, if the ext4 image is generated by the yocto scarthgap with e2fsprogs 1.47, then such an image can be resized only on the host with Ubuntu 23+. The lower versions of Ubuntu have e2fsprogs that can’t resize such an image.

resize is optional. If set to false, it will prevent rouge from resizing the image to the size of the block. This is useful when you want to include a file that is smaller than the block and leave the rest of the block empty.

sparse is optional. If it is set to to false, the raw image will be copied in non-sparse mode. This may increase the final image size on disk and processing time. Use this option only when absolutely necessary, i.e, when some piece of software (like a bootloader) depends on values in unallocated sectors.

Android Sparse Image Block

It is similar to Raw Image Block, but it handles files in Android Sparse image format.

type: android_sparse # defines android sparse block
size: 3000 MiB
image_path: "android/out/target/product/xenvm/userdata.img"

image_path is mandatory. This is a file to be included in the resulting image. rouge will call simg2img2 tool to unpack it before writing it to a resulting image.

size is optional. If it is omitted, rouge will use the data size read from the file. If provided size is smaller than read size, rouge will stop with an error. Thus, you can create a block that is bigger than the unpacked file, but not smaller.

sparse is optional. If it is set to false, Android sparsed image will be completely unsparsed, up to creating a fully mapped file. It is seldom used, but it is added for completeness.

Filesystem Image With Files

This block type allows you to create a new filesystem with some files included from your disk. This is ideal for creating boot partitions, where you store the kernel, initial ramdisk, and so on.

type: ext4 # defines ext4 partition block
size: 30 MiB
items:
  "remote_file1": "path/to/local/file1"
  "remote_file2": "path/to/local/file2"
  "remote_file3": "path/to/local/file3"
  "remote_file4": "path/to/local/file4"
  "remote_dir": "path/to/local/directory/"

type is required. Defines the filesystem type, currently ext4 and vfat are supported. You need to install the mtools package to work with vfat.

items: section is optional. It defines remote:local mapping of files that should be presented on the newly created filesystem. remote part is how the file will be named on the new filesystem, while local is a path on your disk. You can specify parent folders for remote and these folders will be created on the destination filesystem. You may specify not only files but also directories. If the local directory contains subdirectories, they will be created under the remote directory. Older versions of rouge used files: as the name of the section. This name is still possible to use, but it is deprecated. Also, only items: can contain directories.

size is optional. rouge will calculate the total file size and add some space for the filesystem metadata to determine the block size. You can increase the size if you wish.

sparse is optional. If it is set to false, the filesystem image will be copied in non-sparse mode. This may increase the final image size on disk and processing time. Use this option only when absolutely necessary, i.e, when some piece of software (like a bootloader) depends on values in unallocated sectors.

GUID Partition Table (GPT) block

This block type defines GPT along with all partitions. In most cases, this will be your top-level block definition. It can (and should) include other blocks, including other GPT. Inner GPT can come in handy in cases when you are creating an image that holds data for multiple virtual machines and wish to provide each VM with its own GPT.

type: gpt # defines GPT block
partitions:
  boot: # partition label
    gpt_type: 21686148-6449-6E6F-744E-656564454649 # BIOS boot partition (kinda...)
    gpt_guid: 8DA63339-0007-60C0-C436-083AC8230900 # Partition GUID
    type: empty
    size: 30 MiB
  rootfs:
    gpt_type: B921B045-1DF0-41C3-AF44-4C6F280D3FAE # Linux aarch64 root
    type: raw_image
    image_path: "rootfs.ext4"

This example defines GPT with two partitions: boot and rootfs. boot is an empty block and rootfs includes Raw Image block.

partitions: section is mandatory. It defines a list of partitions, where the key is a partition label.

hybrid_mbr forces rouge to create a Hybrid MBR instead of the default Protective MBR. Just so you know, this is an experimental feature, as different OSes handle Hybrid MBR differently; in other words, this type of MBR is not standardized and is not guaranteed to work on your setup. It is mainly added to enable support of older Raspberry PI models, whose bootloader can’t parse GPT. When Hybrid MBR is enabled, the first three partition entries should contain mbr_type property with MBR Partitions type code. In most cases, you will need 0x0C for FAT32 partitions and 0x83 for Linux file systems.

Each partition contains the definition of another block type plus optional keys:

gpt_type: (which we strongly suggest providing) key holds GPT Partition Type GUID. A list of widely used types can be found on Wikipedia, for example.

gpt_guid: key sets the GPT Partition GUID. By default, this GUID is generated automatically to ensure that every partition in the world would have a unique identifier. But there are some cases when external software depends on the exact value of a partition GUID. In such cases, it is possible to hard-code this value. We strongly recommend not to use this key except for the cases when this is necessary because, according to page 121 of Specification the software that makes copies of GPT-formatted disks and partitions must generate new Unique Partition GUID in each GPT Partition Entry.

sector_size is a custom sector size, 512 by default, but some devices (e.g., fancy flash storage) might have a sector of a different size. This key allows tuning for such cases.

mbr_type is used only when hybrid_mbr is set for the GPT block entry. It corresponds to the MBR partition type byte. A list of partition types can be found on Wikipedia.

rouge will place partitions one after another, aligning each partition start at 1 MiB (as per standard recommendation) and partition size to sector size, which defaults to 512 bytes.

Examples

The following example provides multiple different images:

min_ver: 0.3
desc: "rouge sample images"

images:
  empty_image:
    desc: "Just empty 32MB file"
    type: empty
    size: 32 MiB

  unpacked_userdata:
    desc: "Unpacked Android userspace image"
    type: android_sparse
    image_path: "android/out/target/product/xenvm/userdata.img"

  simple_bootable_sd:
    type: gpt
    desc: "Full SD-card/eMMC image"
    partitions:
      boot:
        gpt_type: 21686148-6449-6E6F-744E-656564454649 # BIOS boot partition (kinda...)
        type: ext4
        size: 30 MiB
        items:
          "Image": "yocto/build/tmp/deploy/images/generic-armv8-xt/Image"
          "initrd": "yocto/build/tmp/deploy/images/generic-armv8-xt/uInitrd"
      domd_rootfs:
        gpt_type: B921B045-1DF0-41C3-AF44-4C6F280D3FAE # Linux aarch64 root
        gpt_guid: 8DA63339-0007-60C0-C436-083AC8230900 # Partition GUID
        type: raw_image
        image_path: "yocto/build-domd/tmp/deploy/images/machine/core-image-weston.ext4"

  • rouge sample_images.yaml -i empty_image will generate just an empty file. This is the simplest example.

  • rouge sample_images.yaml -i unpacked_userdata will use simg2img to unpack Android userdata image.

  • rouge sample_images.yaml -i unpacked_userdata will generate a sort of usable image with two GPT partitions: one with data for the bootloader, and the other will contain an ext4 root image created by Yocto.