Merge pull request #1 from jwagner/color-extraction-experiment

Improved getPalette

Author: jwagner

.eslintrc.json

@@ -7,6 +7,7 @@
   },
   "rules": {
     "no-console": "off",
+    "no-continue": "off",
     "no-plusplus": "off",
     "no-bitwise": "off",
     "no-use-before-define": "off",
@@ -24,6 +25,12 @@
       "rules": {
         "import/no-extraneous-dependencies": "off"
       }
+    },
+    {
+      "files": "src/test-helpers.ts",
+      "rules": {
+        "import/no-extraneous-dependencies": "off"
+      }
     }
   ]
 }

README.md

@@ -8,9 +8,19 @@ Dont-crop is a [small](#performance), dependency free javascript library to fit
 
 It can be used to pad images instead of cropping them, for a very compact [blur up](https://engineering.fb.com/2015/08/06/android/the-technology-behind-preview-photos/) and what ever else you can come up with.
 
-![lead image](docs/lead-lossless.webp)
+## Examples
+
+### fitGradient()
+![fitGradient](docs/fitGradient.webp)
 Photo by [Abed Ismail](https://unsplash.com/photos/fZXZ1-hbFrY)
 
+### getPalette()
+![getPalette](docs/getPalette.webp)
+
+### More Examples
+
+View the [demo page](https://29a.ch/sandbox/2021/dont-crop/)  to see more examples and experiment with your own images.
+
 ## Installation
 ```
 npm install -S dont-crop
@@ -68,29 +78,48 @@ It has been tested in:
 The code is reasonably compact and built with tree shaking in mind.
 So your bundles will only include the features you actually use.
 
-When using `fitGradient` only and bundling your code using webpack 5 dont-crop will add about **1.2 kb** to your bundle size.
-`getPalette` will cost you a bit more than **1.4 kb**.
-You can use both for about **2.1 kb**.
+When using `fitGradient` only and bundling your code using webpack 5 dont-crop will add about **1.2 kb** (0.7 gzipped) to your bundle size.
+`getPalette` will cost you a bit more than **3.2 kb** (1.7 gzipped).
+You can use both for about **4 kb** (2 gzipped).
+
+```
+3925 dist/both.js
+1911 dist/both.js.gz
+1264 dist/fitGradient.js
+710  dist/fitGradient.js.gz
+3261 dist/getPalette.js
+1656 dist/getPalette.js.gz
+```
 
 Runtime performance is also fast enough not to worry about.
 
 ```
 # on a AMD Ryzen 9 5950X
-fitGradientToImageData x 43,559 ops/sec ±0.40% (97 runs sampled)
-getPaletteFromImageData x 5,420 ops/sec ±0.21% (92 runs sampled)
+fitGradientToImageData x 19,813 ops/sec ±0.96% (97 runs sampled)
+getPaletteFromImageData(fast=false) x 156 ops/sec ±0.66% (83 runs sampled)
+getPaletteFromImageData(fast=true) x 645 ops/sec ±0.17% (97 runs sampled)
 ```
 
 The versions of the functions operating on images rather than the already downscaled image data are slower.
-Their performance depends on the exact browser and device in question as well but it should generally be in the ballpark of few milliseconds for reasonably sized images.
+Their performance depends on the exact browser and device in question as well but it should generally be in the ballpark of a few milliseconds for reasonably sized images.
 
 ## Test Coverage
 
 The code is well covered in tests. The examples are used as end to end tests in both node and a browser (chrome via puppeteer).
 
+
 ## Algorithms
 
 Glad you asked. `fitGradient()` is using simple [linear regression](https://en.wikipedia.org/wiki/Linear_regression).
-`getPallete()` is based on [k-means](https://en.wikipedia.org/wiki/K-means_clustering).
+
+`getPalete()` is based on [k-means](https://en.wikipedia.org/wiki/K-means_clustering).
+The initial clusters are chosen using a histogram.
+Similar clusters in the result are merged in a post processing step.
+This is necessary because k-means tends to return equally sized clusters
+whereas getPalette is supposed to return distinct clusters.
+The merging is tuned to preserve different hues and colors rather than returning the most prominent shades of color (which might all share a similar hue).
+The processing happens in the CIE Lab color space using CIE76 ΔE*.
+
 
 ## Alternatives
 
@@ -107,6 +136,11 @@ Provides similar functionality to getPalette.
 It weighs in at about 6.4k (version 2.3.2).
 It's been widely used since 2019 so it is definitely more battle proofen.
 From a quick looks it seems to be using median-cut which will likely yield a bit better results than the simplistic k-means used here.
+
+### [fast-average-color](https://github.com/fast-average-color/fast-average-color)
+
+Returns a single average or dominant color color.
+
 
 ### [smartcrop.js](https://github.com/jwagner/smartcrop.js)
 
@@ -117,13 +151,11 @@ to find smarter crops.
 
 There are plenty of interesting ways to improve this library further.
 
-* Adding creative controls over colors (max saturation, max lightness, min lightness)
+* Grouping of colors (saturated, muted, light, dark, warm, cold)
+* Tuning of the variables involved in palette extraction potentially allowing some degree of tweaking by the user of the library
 * Weighting the linear-regression and k-means to focus on the center or edges
 * Using a more robust regression variation like Theil-Senn
 * Gamma corrected linear gradients by manually interpolating the stops
-* Something better than straight k-means for palette extraction
-
-All of these would of course add complexity (and size) as well, so for now it's the simplest thing that could possibly work. 
 
 ## License
 

docs/fitGradient.webp

@@ -1,92 +1,92 @@
 [
   [
-    "#928f82",
-    "#c0cccf",
-    "#30313a",
-    "#57645f"
+    "#a5977d",
+    "#cad5d9",
+    "#55695e",
+    "#594447"
   ],
   [
-    "#1a160e",
-    "#4f3621",
-    "#90542a",
-    "#a77f60"
+    "#aa693a",
+    "#663415",
+    "#504937",
+    "#888173"
   ],
   [
-    "#825b35",
-    "#a07a4a",
-    "#aca699",
-    "#72858c"
+    "#a37c49",
+    "#ae977a",
+    "#a2afb5",
+    "#547c8e"
   ],
   [
-    "#4d7595",
-    "#6c8aa4",
-    "#92a0b2",
-    "#ac9e90"
+    "#4e7696",
+    "#93a1b3",
+    "#ad9780",
+    "#e0dacc"
   ],
   [
-    "#0b1c3a",
-    "#547683",
-    "#4c4f53",
-    "#95a6a6"
+    "#0d2954",
+    "#488a90",
+    "#93826a",
+    "#38281f"
   ],
   [
-    "#13120a",
-    "#2e2617",
-    "#564226",
-    "#85622e"
+    "#90672a",
+    "#5a4322",
+    "#6b5a4e",
+    "#13110a"
   ],
   [
-    "#9a8050",
-    "#b89c74",
-    "#716e32",
-    "#464b26"
+    "#a79749",
+    "#5a6e22",
+    "#bda487",
+    "#343022"
   ],
   [
+    "#bf8a66",
+    "#a47453",
     "#ebeae9",
-    "#ad7b59",
-    "#c28f6e",
-    "#9a6d4d"
+    "#b49580"
   ],
   [
-    "#141619",
-    "#443f3b",
-    "#7d7363",
-    "#c9cec2"
+    "#b07945",
+    "#dde4d9",
+    "#c1a784",
+    "#111519"
   ],
   [
-    "#d8c7d7",
-    "#9595b4",
-    "#116a99",
-    "#312d2c"
+    "#136ea0",
+    "#c5a3be",
+    "#eae3ec",
+    "#463c37"
   ],
   [
-    "#8ab0b4",
-    "#222517",
-    "#41432c",
-    "#64766c"
+    "#8eb4b9",
+    "#484931",
+    "#998d77",
+    "#59726c"
   ],
   [
-    "#131416",
-    "#313741",
-    "#697083",
-    "#a0a7be"
+    "#a1a7bf",
+    "#747c90",
+    "#303442",
+    "#111114"
   ],
   [
-    "#a9acad",
-    "#522215",
-    "#8e6b53",
-    "#3f4b52"
+    "#b2783f",
+    "#a9b0b3",
+    "#436377",
+    "#04151b"
   ],
   [
-    "#cbc3c3",
-    "#e6e2e3",
-    "#423c39",
-    "#91847f"
+    "#ad6536",
+    "#d7d1d2",
+    "#b99c8f",
+    "#433937"
   ],
   [
-    "#573a0e",
-    "#88a3a7",
-    "#ad7c20",
-    "#707a6d"
+    "#e89e10",
+    "#6b640c",
+    "#7badb8",
+    "#4e664b"
   ]
 ]

docs/getPalette.webp

@@ -59,17 +59,35 @@ input, select {
   height: 100%;
 }
 
+.visualization {
+  position: absolute;
+  z-index: 10;
+  top:  0em;
+  right: 0;
+  /* transform: scale(0.5); */
+  /* transform-origin: 100% 0%; */
+  transition: opacity 0.2s ease-out;
+  opacity: 0;
+}
+
+.frame:hover  .visualization {
+  opacity: 1.0;
+}
+
 .colors {
   display: inline-block;
   margin-right: 1ex;
-  border: 1px solid white;
-  height: 1em;
+  height: 2em;
+  vertical-align: bottom;
 }
 
 .color {
   display: inline-block;
-  width: 1em;
-  height: 1em;
+  width: 2em;
+  height: 2em;
+  border: 2px solid white;
+  border-radius: 100%;
+  margin-right: 1ex;
 }
 
 .dropzone {
@@ -95,6 +113,9 @@ input, select {
   box-sizing: border-box;
   display: flex;
   align-items: center;
-  justify-content: center;
-  
+  justify-content: center; 
+}
+
+.info {
+  margin: 1em 0;
 }

examples/node-sharp/snapshots/getPaletteFromImageData.json

@@ -22,9 +22,9 @@ <h2>Test it with our own image</h2>
       <span class="note">Drop an image onto this frame or click it to select one.</span>
       <input class="dropzone" type="file" />
     </div>
-    <p class="info">
-    <div class="colors"></div>
-    </p>
+    <div class="info">
+      <div class="colors"></div>
+    </div>
   </div>
   <h2>Or a few test images</h2>
   <%= htmlWebpackPlugin.tags.bodyTags %>