Allow image size specification using any SI or IEC unit #310
Conversation
|
Couple of notes:
|
In clalancette/oz#310 I'm proposing we change the unit of the `generate_diskimage` method's size arg from gibibytes to bytes. This needs adjusting to match that. Don't merge this unless and until that PR is merged. Signed-off-by: Adam Williamson <[email protected]>
|
ImageFactory PR: redhat-imaging/imagefactory#458 |
|
FWIW, I'm also working on a PR that fixes a few problems, gets the test suite (and hence oz itself) working on Python 3.12, and - hopefully - a github action to run it automatically, in containers (I have the test suite working in containers locally, just need to wrap it up). |
AdamWill
force-pushed
the
image-size-bytes
branch
5 times, most recently
from
edcc28a
to
3e95bad
Compare
AdamWill
changed the title
|
hmm, I think this failure might actually be a real one! looking into it. |
AdamWill
force-pushed
the
image-size-bytes
branch
from
3e95bad
to
014411b
Compare
|
yaaay. The change to the XML schema is probably a bit more permissive than it should be, but, you know, regexes. i don't want to make it too complicated. It turns out we do need this, because an attempt to build a current Fedora Workstation aarch64 image with size 14 gibibytes - the largest currently-possible size that's less than 16 gigabytes - failed because it was 331 MB short. This should let us specify the size as 16 gigabytes and get a successful, not-too-large image. I will do a scratch build of oz with this patch and get Kevin to do a test image build. |
|
This scratch build was done with size set to |
|
BTW, to resolve the licensing concern, Richard said the function is too trivial to be copyrightable. |
Currently, oz only allows image sizes to be specified as integer amounts of gibibytes or tebibytes (that's IEC power-of-two units). This is unfortunately inflexible. Consider that storage media are typically specified in SI power-of-ten sizes, so a USB stick may be 16 gigabytes (SI power-of-ten GB) in size - that's 16,000,000,000 bytes. Let's say we want to build an image that will fit on such a USB stick. oz will only allow us to build an image of 15,032,385,536 bytes (14 gibibytes) or 16,106,127,360 bytes (15 gibibytes). So we're either slightly too big, or leaving nearly a gigabyte on the table. This allows the image size to be specified in the TDL with most any IEC or SI unit suffix, from B (bytes) all the way up to YiB (yobibytes). A size with no suffix or the suffix "G" is taken as gibibytes and a size with the suffix "T" is taken as tebibytes, as before, but other ambiguous suffixes are not accepted. All casing is accepted. Behind the scenes, we convert the size to bytes and specify it that way in the libvirt XML when creating the image in _internal_generate_diskimage. This does change the interface of generate_diskimage(), by making the unit for the size argument bytes instead of gibibytes. I can't see a clean way to avoid this while allowing flexibility. I have checked, and AFAICT, among active projects, only oz itself and the ImageFactory TinMan plugin call this function. The TinMan plugin will need a trivial change to its fallback default value. Signed-off-by: Adam Williamson <[email protected]>
AdamWill
force-pushed
the
image-size-bytes
branch
from
014411b
to
b609f75
Compare
|
just to make things clearer, I've moved the CI fixes to #314 . that means CI fails on this PR now, but with 314 merged it would pass again. |
|
It's been two weeks. This is a blocker for Fedora 40 Beta (https://bugzilla.redhat.com/show_bug.cgi?id=2263771), so it'd be nice to get this resolved… |
|
We don't necessarily need it merged upstream, though. ;) In fact, the builders have it already, as part of this update. What we don't have yet is a patched Koji, Kevin and I can work on that next week I guess. |
This has never been necessary, because oz has always treated a size with no unit as being in gibibytes. After clalancette/oz#310 it will be actively harmful, because it will prevent us from using that new ability of oz to specify a size using any other unit, as we have a specific reason to want: https://bugzilla.redhat.com/show_bug.cgi?id=2247611 Signed-off-by: Adam Williamson <[email protected]>
This has never been necessary, because oz has always treated a size with no unit as being in gibibytes. After clalancette/oz#310 it will be actively harmful, because it will prevent us from using that new ability of oz to specify a size using any other unit, as we have a specific reason to want: https://bugzilla.redhat.com/show_bug.cgi?id=2247611 Signed-off-by: Adam Williamson <[email protected]>
Currently, oz only allows image sizes to be specified as integer amounts of gibibytes or tebibytes (that's IEC power-of-two units). This is unfortunately inflexible. Consider that storage media are typically specified in SI power-of-ten sizes, so a USB stick may be 16 gigabytes (SI power-of-ten GB) in size - that's 16,000,000,000 bytes. Let's say we want to build an image that will fit on such a USB stick. oz will only allow us to build an image of 15,032,385,536 bytes (14 gibibytes) or 16,106,127,360 bytes (15 gibibytes). So we're either slightly too big, or leaving nearly a gigabyte on the table.
This allows the image size to be specified in the TDL with most any IEC or SI unit suffix, from B (bytes) all the way up to YiB (yobibytes). A size with no suffix or the suffix "G" is taken as gibibytes and a size with the suffix "T" is taken as tebibytes, as before, but other ambiguous suffixes are not accepted. All casing is accepted. Behind the scenes, we convert the size to bytes and specify it that way in the libvirt XML when creating the image in _internal_generate_diskimage.
This does change the interface of generate_diskimage(), by making the unit for the size argument bytes instead of gibibytes. I can't see a clean way to avoid this while allowing flexibility. I have checked, and AFAICT, among active projects, only oz itself and the ImageFactory TinMan plugin call this function. The TinMan plugin will need a trivial change to its fallback default value.
This is currently tagged WIP because I am not sure it's safe to include the sizeof_fmt snippet; its licensing is complicated. I'm asking Richard Fontana for his wisdom on this ATM. If it's not safe to include we'll have to rewrite it or just print the size in bytes or something.