fix: tools/toFixed precision #1950

carlosjeurissen · 2024-02-02T22:26:26Z

Currently the method meant to reduce the amount of decimals (lib/tools/toFixed) sometimes increases the amount of decimals due to float handling in JavaScript. This PR addresses this by checking the original amount of decimals and comparing it to the requested precision. If it already meets the precision requested, it will simply return the original number.

There are more areas in which float issues appear. Potentially we might want to start using some kind of BigNumber library to tackle this as to be able to do more lossless operations and to allow scale-based optimisations as proposed here: #1270

KTibow · 2024-02-03T01:33:50Z

This reduces performance every time it's called since toString is slow. I also don't see how it fixes #1944.

carlosjeurissen · 2024-02-03T08:55:08Z

@KTibow Thanks for your quick reply! The performance impact is reduced as much as possible by skipping the whole fixing and toString conversion when it is not needed.

While agree calling this method can take a bit more time, it is key to SVGO to the fixing correctly. It is unforgiving SVGO can end up with bigger float numbers in output svg files than what came in. A very low floatPrecision should not be required to overcome this.

As for how it fixes #1944, I experienced cases in which SVGO replaced floats in pathdata like 30.5779623 into 30.577962299999996. This PR addresses this and this seems what #1944 is talking about as well.

KTibow · 2024-02-03T17:51:33Z

The performance impact is reduced as much as possible by skipping the whole fixing and toString conversion when it is not needed.

In the vast majority of cases, it won't be an integer, so it will still be called.

I experienced cases in which SVGO replaced floats in pathdata like 30.5779623 into 30.577962299999996.

Can you give me a repro for this?

this seems what #1944 is talking about as well.

Don't think it will. In fact toFixed will never be called as floatPrecision is undefined:

const roundAndStringify = (number, precision) => {
  if (precision != null) {
    number = toFixed(number, precision);
  }

  return {
    roundedStr: removeLeadingZero(number),
    rounded: number,
  };
};

carlosjeurissen · 2024-02-03T19:27:54Z

In the vast majority of cases, it won't be an integer, so it will still be called.

Yet it will still skip the fixing itself when the amount of decimals is within bounds.

Can you give me a repro for this?

Sure! Config being:

{
  multipass: true,
  floatPrecision: 19,
  plugins: [
    'preset-default'
  ]
}

Input file:

<?xml version="1.0" encoding="utf-8"?><svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48"><g fill="none" fill-rule="evenodd"><path fill="#FBBC04" fill-rule="nonzero" d="M24,39 L24.0008882,44.4879866 C24.0008882,45.3604302 23.2584362,46.0528855 22.3884924,45.9903895 C11.4841973,45.2029404 2.79225918,36.5035028 2.00730992,25.609207 C1.94231412,24.7367634 2.63476936,23.9943114 3.50721296,23.9943114 L9,24 L24,39 Z"/><path fill="#EA4335" fill-rule="nonzero" d="M24.0008882,9 L24.0008882,3.50063618 C24.0008882,2.62819257 24.7433402,1.93573734 25.613284,1.9982333 C36.5075797,2.78568239 45.2070174,11.48512 45.9944665,22.3794158 C46.0569624,23.2493596 45.3645072,23.9918116 44.4920636,23.9918116 L39,24 L24.0008882,9 Z"/><path fill="#5BB974" fill-rule="nonzero" d="M30.5779623,23.9973355 L34.9999999,19.9999964 L39.7873677,23.9943114 L39.7873677,24.0343088 C39.7873677,32.6251864 32.9087151,39.6086058 24.3583215,39.7776555 L24.0008882,39.7807909 L18.9999999,34.9999964 L23.9998801,30.5713862 C27.6322982,30.5713862 30.5763286,27.6283637 30.5779623,23.9973355 Z"/><path fill="#669DF6" fill-rule="nonzero" d="M24.0008882,8.20783189 L29,13 L23.9994087,17.4168319 L23.7699875,17.4212139 C20.3228427,17.5401494 17.5495252,20.3118234 17.4279689,23.7583018 L17.4234087,23.9978319 L13,29.0000008 L8.21440868,23.9943114 L8.21440868,23.954314 C8.21440868,15.3634364 15.0930613,8.380017 23.6434549,8.21096734 L24.0008882,8.20783189 Z"/><path fill="#1967D2" fill-rule="nonzero" d="M24.0008882,8.20783189 L24.050885,8.20783189 C32.6368238,8.20783189 39.6153045,15.081606 39.7842345,23.6270617 L39.7873677,23.9943114 L30.577963,23.9943114 C30.577963,20.3618933 27.6333063,17.4172366 24.0008882,17.4172366 L24.0008882,8.20783189 Z"/><path fill="#1E8E3E" fill-rule="nonzero" d="M17.4238134,23.9943114 C17.4238134,27.6267295 20.3684701,30.5713862 24.0008882,30.5713862 L24.0008882,39.7807909 L23.9858892,39.7807909 C15.2739523,39.7807909 8.21440868,32.7137477 8.21440868,23.9943114 L17.4238134,23.9943114 Z"/><g fill="#D8D8D8" stroke="#979797" transform="translate(25, 25) rotate(45) translate(-25, -25)translate(21, 21)"><circle cx="3" cy="3" r="2.5"/><circle cx="5" cy="5" r="2.5"/></g></g></svg>

Before PR: 1969 characters
After PR: 1936 characters

Don't think it will. In fact toFixed will never be called as floatPrecision is undefined

I still was able to reduce the file size with the above config.
Before PR: 18,321 bytes
After PR: 18,051 bytes

KTibow · 2024-02-03T19:55:19Z

Ah, I see. I'm tempted to say that this would be better implemented by just returning the original number or falling back to Number(.toFixed()) when floatPrecision is more than 16, as at that point you start getting into the "not safely representable" territory anyway. It also keeps the speed high.

Speed comparison on some synthetic data w/ jsbench:

original: ~12,000,000 ops/s
yours: ~380,000 ops/s
mine (usually): ~12,000,000 ops/s
mine (19 w/ returning original): ~10,000,000 ops/s
mine (19 w/ Number(.toFixed())): ~597,000 ops/s

Of note, the old method of parsing the number is actually faster than your way of checking the decimal count. String manipulation is expensive!

I still was able to reduce the file size with the above config.

I see that your change helps in many scenarios where you set floatPrecision 19, but my issue wasn't about the scenario of an explicit float precision or a very high one, it was a simple issue where the fallback was missing, solved by #1945. It's still worth discussing other methods to handle this behavior with extremely high float precisions though.

…n 17

carlosjeurissen · 2024-02-03T20:31:32Z

Ah, I see. I'm tempted to say that this would be better implemented by just returning the original number or falling back to Number(.toFixed()) when floatPrecision is more than 16, as at that point you start getting into the "not safely representable" territory anyway. It also keeps the speed high.

Fair. I have updated the PR to just do a check on precision higher than 17. As JavaScript drops any precision at that point anyway. However, this means the issue of toFixed returning more decimals will not be fixed when the precision is set lower than 17. Potentially some option could be added to specifically do a decimal check to squeeze the last bits out of the file. However this can be addressed in another PR.

Of note, the old method of parsing the number is actually faster than your way of checking the decimal count. String manipulation is expensive!

The String.prototype.toFixed method can at times incorrectly round numbers. For example, 21.0565 is rounded to 21.056 with a precision of 3.

I see that your change helps in many scenarios where you set floatPrecision 19, but my issue wasn't about the scenario of an explicit float precision or a very high one, it was a simple issue where the fallback was missing, solved by #1945. It's still worth discussing other methods to handle this behavior with extremely high float precisions though.

Fair. Just tested with a normal floatPrecision and saw no difference. Updated the PR to reflect this.

…umericValues

carlosjeurissen · 2024-02-04T16:01:57Z

@KTibow Updated the PR to use a variation of the toFixed methods in specific places which is string based. As these situations were already using strings there seems to be no real performance lost.

carlosjeurissen · 2024-02-19T16:19:52Z

@SethFalco Let me know on how to move forward with this one. Or if there alternative paths to improve the precision situation.

SethFalco · 2024-02-19T21:12:50Z

@carlosjeurissen Hey! Thanks for opening the PR, and thanks for your patience! I'll be with you later this week with the review.

carlosjeurissen · 2024-06-25T13:09:40Z

@SethFalco a friendly reminder of this PR. If you are still interested I can look into the merge conflict.

GreLI · 2024-06-27T20:35:32Z

lib/svgo/tools.js

+  const result = fixed.toString();
+
+  // prevent returning more digits than originally given
+  const isRegression = result.length > numStr.length;


toFixed(2.5845, 3) correctly gives me 2.585. Are there real examples where the check is needed? I iterated over every number few years ago and didn't found any.

toFixed(1.99999999999, 18) -> 1.9999999999899998

@GreLI Here is an example of before and after which adds additional digits as the result of floating point arithmetic.

Before:

<?xml version="1.0" encoding="UTF-8"?> <svg width="48px" height="48px" viewBox="0 0 48 48" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"> <defs> <linearGradient x1="31.7538776%" y1="65.3789342%" x2="76.9899438%" y2="27.2412226%" id="linearGradient-1"> <stop stop-color="#217BFE" offset="0%"></stop> <stop stop-color="#078EFB" offset="27%"></stop> <stop stop-color="#A190FF" offset="78%"></stop> <stop stop-color="#BD99FE" offset="100%"></stop> </linearGradient> </defs> <g stroke="none" stroke-width="1" fill="none" fill-rule="evenodd"> <path d="M45.912505,23.9125099 C42.8951764,23.9125099 40.1078348,23.340075 37.4554855,22.2152028 C34.8006363,21.0528349 32.4607693,19.4655153 30.4983808,17.5032383 C28.5359923,15.5409613 26.9510823,13.1987274 25.7861485,10.5465288 C24.6587126,7.89433019 24.088745,5.1046472 24.088745,2.08749006 C24.088745,2.03999546 24.0512471,2 24.0012499,2 C23.9512528,2 23.9137549,2.03999546 23.9137549,2.08749006 C23.9137549,5.1046472 23.3237884,7.89183047 22.1613545,10.5465288 C21.0339185,13.2012271 19.4665076,15.5409613 17.5041191,17.5032383 C15.5417306,19.4655153 13.1993637,21.0503352 10.5470144,22.2152028 C7.89466508,23.3425747 5.10482359,23.9125099 2.08749503,23.9125099 C2.03999773,23.9125099 2,23.9525054 2,24 C2,24.0474946 2.03999773,24.0874901 2.08749503,24.0874901 C5.10482359,24.0874901 7.89216522,24.677423 10.5470144,25.8397909 C13.2018635,26.9671628 15.5417306,28.5344847 17.5041191,30.4967617 C19.4665076,32.4590387 21.0339185,34.8012726 22.1613545,37.4559709 C23.3237884,40.1081695 23.9137549,42.8953528 23.9137549,45.9125099 C23.9137549,45.9600045 23.9537526,46 24.0012499,46 C24.0487472,46 24.088745,45.9625043 24.088745,45.9125099 C24.088745,42.8953528 24.6587126,40.1081695 25.7861485,37.4559709 C26.9485825,34.8012726 28.5334924,32.4615385 30.4983808,30.4967617 C32.4607693,28.5344847 34.8006363,26.9671628 37.4554855,25.8397909 C40.1078348,24.677423 42.8951764,24.0874901 45.912505,24.0874901 C45.9600023,24.0874901 46,24.0499943 46,24 C46,23.9500057 45.9600023,23.9125099 45.912505,23.9125099 Z" fill="url(#linearGradient-1)" fill-rule="nonzero"></path> </g> </svg>

after:

<?xml version="1.0" encoding="utf-8"?> <svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48"> <defs> <linearGradient id="a" x1="31.753877599999996%" x2="76.9899438%" y1="65.3789342%" y2="27.2412226%"> <stop offset="0%" stop-color="#217BFE"/> <stop offset="27%" stop-color="#078EFB"/> <stop offset="78%" stop-color="#A190FF"/> <stop offset="100%" stop-color="#BD99FE"/> </linearGradient> </defs> <path fill="url(#a)" d="M45.912505 23.9125099C42.8951764 23.9125099 40.1078348 23.340075 37.4554855 22.2152028 34.8006363 21.0528349 32.4607693 19.4655153 30.4983808 17.5032383 28.5359923 15.5409613 26.9510823 13.1987274 25.7861485 10.5465288 24.6587126 7.89433019 24.088745 5.1046472 24.088745 2.0874900600000004 24.088745 2.03999546 24.0512471 2 24.0012499 2 23.9512528 2 23.9137549 2.03999546 23.9137549 2.08749006 23.9137549 5.1046472 23.3237884 7.89183047 22.1613545 10.5465288 21.0339185 13.2012271 19.4665076 15.5409613 17.5041191 17.5032383 15.5417306 19.4655153 13.1993637 21.0503352 10.5470144 22.2152028 7.89466508 23.3425747 5.10482359 23.9125099 2.0874950299999995 23.9125099 2.03999773 23.9125099 2 23.9525054 2 24S2.03999773 24.0874901 2.08749503 24.0874901C5.10482359 24.0874901 7.89216522 24.677423 10.5470144 25.8397909 13.2018635 26.9671628 15.5417306 28.5344847 17.5041191 30.4967617 19.4665076 32.4590387 21.0339185 34.8012726 22.1613545 37.4559709 23.3237884 40.1081695 23.9137549 42.8953528 23.9137549 45.9125099 23.9137549 45.9600045 23.9537526 46 24.0012499 46S24.088745 45.9625043 24.088745 45.9125099C24.088745 42.8953528 24.6587126 40.1081695 25.7861485 37.4559709 26.9485825 34.8012726 28.5334924 32.4615385 30.4983808 30.4967617 32.4607693 28.5344847 34.8006363 26.9671628 37.4554855 25.8397909 40.1078348 24.677423 42.8951764 24.0874901 45.912505 24.0874901 45.9600023 24.0874901 46 24.0499943 46 24S45.9600023 23.9125099 45.912505 23.9125099"/> </svg>

with the following svgo config:

const prettyprint = true; const floatPrecision = 19; const js2svg = { eol: 'lf', finalNewline: true, indent: 2, pretty: prettyprint, }; const config = { multipass: true, js2svg: js2svg, floatPrecision: floatPrecision, plugins: [ { name: 'preset-default', params: { overrides: { // customize default plugin options inlineStyles: { onlyMatchedOnce: false, }, convertPathData: { makeArcs: { threshold: 2.5, tolerance: 0.5, }, transformPrecision: floatPrecision, forceAbsolutePath: false, noSpaceAfterFlags: false, }, cleanupNumericValues: { }, convertShapeToPath: { convertArcs: false, }, removeUnknownsAndDefaults: { keepAriaAttrs: false, keepDataAttrs: false, }, mergePaths: { force: true, noSpaceAfterFlags: true, }, sortAttrs: { order: [ 'id', 'fill-rule', 'fill', 'stroke', 'stroke-width', 'width', 'height', 'x', 'x1', 'x2', 'y', 'y1', 'y2', 'cx', 'cy', 'r', 'marker', 'd', 'points', ], xmlnsOrder: 'front', }, removeDoctype: false, removeViewBox: false, removeXMLProcInst: false, }, }, }, 'convertStyleToAttrs', 'removeDimensions', 'removeOffCanvasPaths', ], }; export default config;

As you can see it happens with the x1 attribute in the linear-gradient. And there is also the 2.0874900600000004 in a d attribute further down.

@KTibow Ah, I see. Without if (precision > 17 || num % 1 == 0) return num; on line 237 it really have such issues.

Though I must say, that are quite extreme precisions, far from practical ones. E.g. you lose precision by multiplication numbers with more than 9 digits in the fractional part due to how floating point calculations work. I wouldn't recommend to use floatPrecision = 19 to anyone. Editors usually don't write more than 5 fractional digits.

Speaking of pixels there wouldn't be difference more than 1/256 of pixel (ok, 1/512 with high dpi display). So, given the usual display has no more than thousands of pixels, 7 digits in total is more than enough. Thus there is no sense in more than 5 fractional digits in percentages. Practically, one wouldn't see difference for something like SVGO default 3 digits. For some images it can be even 2 or 1 (without curves—I have some ideas about that).

The most egregious example I've seen in the wild is:

<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 673 205"> <path d="M0 0L6235704.138785 2260595.537351" transform="matrix(0.0000431728,0,0,0.0000431865,60.47463,20.8221)" stroke-width="12700" stroke="black"/> </svg>

— #1858 (comment)

Without any kind of path normalization to bring the scale of the numbers closer to 0 (which may be worth looking at in future) reducing the precision can be problematic. But even then, a float precision like 19 does seem odd.

So, given the usual display has no more than thousands of pixels, 7 digits in total is more than enough

Generally this is true, but depend on the scale that transforms operate on, that can still cause problems. (Not denying, that SVGs like the example given seem bizarre, but it does happen.)

I haven't reviewed this PR yet as I've been focused on other things in SVGO, so can't say if I'm in favor of the change or not yet. Just noting that it still may be worth having better support for large precision values if it's otherwise causing problems.

@carlosjeurissen Sorry for putting you off! I've been busy with client work, and now that I have time for SVGO again, my priority is with issues related to the v4 release candidates. (Namely imports/interface related issues.)

Once those are sorted, we can take a deeper look at this.

Yeah, transformations are often sick and overused. That's why it's better to apply transformations at first—that makes thinks easier.

carlosjeurissen added 2 commits February 2, 2024 23:21

Prevent toFixed from adding more decimals than was given

840ef2d

Use toFixed in more plugins

5b48095

carlosjeurissen changed the title ~~Fix/to fixed precision~~ fix: tools/toFixed precision Feb 2, 2024

Simplify logic by just skipping toFixed with any precision higher tha…

90927e2

…n 17

carlosjeurissen added 4 commits February 4, 2024 16:59

Add toFixedStr as string alternative for toFixed

1d389d2

Start using new toFixedStr method in cleanupListOfValues and cleanupN…

48e3d1c

…umericValues

Correct return type for toFixedStr

89ae4bb

Start using toFixedStr in path.js

6d1a6b2

GreLI reviewed Jun 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: tools/toFixed precision #1950

fix: tools/toFixed precision #1950

carlosjeurissen commented Feb 2, 2024 •

edited

Loading

KTibow commented Feb 3, 2024

carlosjeurissen commented Feb 3, 2024

KTibow commented Feb 3, 2024 •

edited

Loading

carlosjeurissen commented Feb 3, 2024

KTibow commented Feb 3, 2024 •

edited

Loading

carlosjeurissen commented Feb 3, 2024

carlosjeurissen commented Feb 4, 2024

carlosjeurissen commented Feb 19, 2024

SethFalco commented Feb 19, 2024

carlosjeurissen commented Jun 25, 2024

GreLI Jun 27, 2024 •

edited

Loading

KTibow Jun 27, 2024 •

edited

Loading

carlosjeurissen Jun 27, 2024

GreLI Jun 27, 2024 •

edited

Loading

SethFalco Jun 28, 2024 •

edited

Loading

GreLI Jun 28, 2024

fix: tools/toFixed precision #1950

Are you sure you want to change the base?

fix: tools/toFixed precision #1950

Conversation

carlosjeurissen commented Feb 2, 2024 • edited Loading

KTibow commented Feb 3, 2024

carlosjeurissen commented Feb 3, 2024

KTibow commented Feb 3, 2024 • edited Loading

carlosjeurissen commented Feb 3, 2024

KTibow commented Feb 3, 2024 • edited Loading

carlosjeurissen commented Feb 3, 2024

carlosjeurissen commented Feb 4, 2024

carlosjeurissen commented Feb 19, 2024

SethFalco commented Feb 19, 2024

carlosjeurissen commented Jun 25, 2024

GreLI Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

KTibow Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

carlosjeurissen Jun 27, 2024

Choose a reason for hiding this comment

GreLI Jun 27, 2024 • edited Loading

Choose a reason for hiding this comment

SethFalco Jun 28, 2024 • edited Loading

Choose a reason for hiding this comment

GreLI Jun 28, 2024

Choose a reason for hiding this comment

carlosjeurissen commented Feb 2, 2024 •

edited

Loading

KTibow commented Feb 3, 2024 •

edited

Loading

KTibow commented Feb 3, 2024 •

edited

Loading

GreLI Jun 27, 2024 •

edited

Loading

KTibow Jun 27, 2024 •

edited

Loading

GreLI Jun 27, 2024 •

edited

Loading

SethFalco Jun 28, 2024 •

edited

Loading