Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
77 commits
Select commit Hold shift + click to select a range
1a79731
Spike; create master crop file via vips.
tonytw1 May 21, 2025
6576da9
Naive resize crops to jpegs only step.
tonytw1 May 23, 2025
8fc2a6e
createCrops and resizeImageVips in same Future and arena to allow sha…
tonytw1 May 23, 2025
366dacb
Try to use 1 load of the source image across all resizes; do the resi…
tonytw1 May 23, 2025
1c81f44
createCrops takes the masterCrop as a VImage rather than a file.
tonytw1 May 23, 2025
e8b4bfd
createMasterCrop moving to same arena.
tonytw1 May 23, 2025
68550fc
Remove colourModel check from createMasterCrop.
tonytw1 May 23, 2025
76453ac
Logging. Show when we reload the master crop from disk.
tonytw1 May 23, 2025
df2cfa9
Correct arena close timing.
tonytw1 May 23, 2025
7b06ea3
createMasterCrop no Futures.
tonytw1 May 23, 2025
1a7a539
arena scope?
tonytw1 May 23, 2025
1849013
arena scope?
tonytw1 May 23, 2025
5b825f8
arena scope?
tonytw1 May 23, 2025
b7bd8ec
arena scope?
tonytw1 May 23, 2025
2c83a62
Spike; no reload of master crop image.
tonytw1 May 23, 2025
df0f9c9
Marking TODO; why local file for master crop.
tonytw1 May 24, 2025
0cf8745
Log local resize file location.
tonytw1 May 24, 2025
3f446f5
PNG specific optimiseImage steps can move straight into the resizeIma…
tonytw1 May 24, 2025
a536e35
Logging.
tonytw1 May 24, 2025
f34ffd4
TODO Strip true is needed to stop exif auto correction until we can s…
tonytw1 May 24, 2025
63e372f
Revert "Revert "Master image converted to sRBG colour space.""
tonytw1 May 24, 2025
05c8c9d
Resize uses save file.
tonytw1 Feb 10, 2026
8fe1e40
Log crop type decision.
tonytw1 May 24, 2025
ee7e239
Colour correct not effective if metadata stripped. Bake it in?
tonytw1 May 24, 2025
dc921b9
Refactor; master crop image file write pushes up out of the create ma…
tonytw1 May 24, 2025
f4e3352
createMasterCrop uses saveToFile and can now save PNGs.
tonytw1 Feb 10, 2026
ebf6089
Refactor. S3 store of master crop pushes up. MasterCrop is a simpler …
tonytw1 May 24, 2025
b5ed929
Refactor. File create for mastercrop pushes up to be next to the s3 s…
tonytw1 May 24, 2025
d82cda7
Master file create order; before arena close.
tonytw1 May 24, 2025
15099d5
Comment; master crop if different from the the full dimensions crop; …
tonytw1 May 24, 2025
e05ef65
Refactor; split creation of crop files from publishing to S3.
tonytw1 May 25, 2025
9350e04
quantise crops.
tonytw1 Feb 11, 2026
d3631be
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
d66b744
isGraphic was sendint all TIFFs to PNG.
tonytw1 May 25, 2025
ce2464a
Correct CMYK renders too light in crops.
tonytw1 May 25, 2025
02d06c2
Unused parameters.
tonytw1 May 26, 2025
3bf9780
Unused parameters.
tonytw1 May 26, 2025
e30df05
Bypass icc_transform for LAB images.
tonytw1 May 26, 2025
6fb0148
Remove non used non vips functions.
tonytw1 Feb 28, 2026
2722202
Remove non used runConvert command.
tonytw1 May 26, 2025
63de61d
Remove unused iccColourSpace val.
tonytw1 Jun 29, 2025
983921c
Reapply png master quality.
tonytw1 Jun 29, 2025
79067d5
Crop quality values can be int.
tonytw1 Jan 17, 2026
db9735d
CropType decision can be deferred until after the master crop has bee…
tonytw1 Nov 24, 2025
194586c
Refactor. Push the isGraphic? decision up out of cropType so that is …
tonytw1 Dec 29, 2025
ac36103
Spike. vips based implementation of isGraphic? is likely to involve t…
tonytw1 Dec 29, 2025
a3f2fdb
Test to exercise ImageOperations resize.
tonytw1 Feb 18, 2026
04c9d78
[libvips] Testing around hasAlpha.
tonytw1 Feb 1, 2026
6423db6
Clarify master crop quality values.
tonytw1 Jan 17, 2026
be05978
[libvips-cropping] Allow embedded icc profile to be used in crop icc …
tonytw1 Jan 19, 2026
3798d9f
[libvips-cropping] Image operations resize takes the output file as a…
tonytw1 Jan 21, 2026
601c738
[libvips-cropping] Cropping of LAB fixed by making non icc_transform …
tonytw1 Jan 19, 2026
b7e51b8
[libvips-cropping] Show that it is possible to manually trim metadata…
tonytw1 Feb 12, 2026
992b4bc
[libvips-cropping] Bake the credit, copyright and supplier transmissi…
tonytw1 Jan 24, 2026
57a2c29
XMP Credit -> photoshop:Credit
tonytw1 Feb 24, 2026
3be83e1
XMP Byline -> creator.
tonytw1 Feb 24, 2026
731accc
[libvips-cropping] While manually stripping metadata might be useful,…
tonytw1 Feb 12, 2026
a8e3b7d
[libvips-cropping] Additional resize tests.
tonytw1 Jan 31, 2026
e084f14
[libvips-cropping] Test for correct resize of png with alpha.
tonytw1 Feb 16, 2026
da39444
[libvips-cropping] Provide an example of the vips can't render a LAB …
tonytw1 Jan 31, 2026
67aad0a
[libvips-cropping] Cleanup; Lower the input of apiSource to imageId f…
tonytw1 Jan 26, 2026
e54fa59
[libvips-cropping] resizeImageVips does not need the MasterCrop object
tonytw1 Mar 15, 2026
acb64e8
[libvips-cropping] Metadata for crop call.
tonytw1 Feb 1, 2026
4958db2
[libvips-cropping] Crop metadata in XMP via vips removes exiftool as …
tonytw1 Jan 24, 2026
2ada421
[libvips-cropping] createCrops moves to image operations for testing.
tonytw1 Jan 26, 2026
187e11e
[libvips-cropping] Tests to exercise crops.
tonytw1 Jan 31, 2026
fec63b9
[libvips-cropping] createCrops can execute it's resizeImage calls in …
tonytw1 Mar 15, 2026
7ec4807
[libvips-cropping] Master crop save in parallel.
tonytw1 Jan 26, 2026
14891a9
[libvips-cropping] Logging.
tonytw1 Jan 26, 2026
96714bd
[libvips-cropping] Logging and imports.
tonytw1 Jan 26, 2026
481c5e3
[libvips-cropping] Do not send resizes until master has successfully …
tonytw1 Jan 27, 2026
63ae41f
[libvips-cropping] getImageInformation uses imageOperations.hasAlpha …
tonytw1 Feb 1, 2026
76d0d31
[libvips-cropping] Missing arena.close()
tonytw1 Feb 24, 2026
c7a91ea
[libvips-cropping] Crops uses master hasAlpha image operation rather …
tonytw1 Feb 1, 2026
1354eaa
[libvips-cropping] Do not repeat calls to image_get_interpretation
tonytw1 Feb 24, 2026
ef8caff
Removes exiftool from base image.
tonytw1 Feb 20, 2026
3feb2b0
Remove pnquant from base image.
tonytw1 Jan 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ lazy val commonLib = project("common-lib").settings(
// declare explicit dependency on desired version of aws sdk v2 bedrock runtime
"software.amazon.awssdk" % "bedrockruntime" % awsSdkV2Version,
"software.amazon.awssdk" % "s3vectors" % awsSdkV2Version,
"com.adobe.xmp" % "xmpcore" % "6.1.11",
ws,
"org.testcontainers" % "testcontainers-elasticsearch" % "2.0.2" % Test,
),
Expand Down

Large diffs are not rendered by default.

This file was deleted.

Original file line number Diff line number Diff line change
Expand Up @@ -87,14 +87,6 @@ object ImageMagick extends GridLogging {
op
}

def runConvertCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[Unit] = {
Stopwatch.async(s"Using ${if(useImageMagick) "imagemagick" else "graphicsmagick"} for imaging conversion operation '$op'") {
Future {
new ConvertCmd(!useImageMagick).run(op)
}
}
}

def runIdentifyCmd(op: IMOperation, useImageMagick: Boolean)(implicit logMarker: LogMarker): Future[List[String]] = {
Stopwatch.async(s"Using ${if (useImageMagick) "imagemagick" else "graphicsmagick"} for imaging identification operation '$op'") {
Future {
Expand Down
Binary file not shown.
25 changes: 25 additions & 0 deletions common-lib/src/test/resources/schaik.com_pngsuite/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
Test PNG images
===============

These images are taken from http://www.schaik.com/pngsuite/pngsuite_bas_png.html

basn0g08 - 8 bit (256 level) grayscale
basn2c08 - 3x8 bits rgb color
basn3p08 - 8 bit (256 color) paletted
basn6a08 - 3x8 bits rgb color + 8 bit alpha-channel

LICENCE
-------

At the time of downloading these images the licence file at http://www.schaik.com/pngsuite/PngSuite.LICENSE contained the following text:

```
PngSuite
--------

Permission to use, copy, modify and distribute these images for any
purpose and without fee is hereby granted.


(c) Willem van Schaik, 1996, 2011
```
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -1,21 +1,18 @@
package com.gu.mediaservice.lib.imaging

import app.photofox.vipsffm.jextract.VipsRaw
import app.photofox.vipsffm.{VImage, Vips}
import app.photofox.vipsffm.Vips
import com.gu.mediaservice.lib.BrowserViewableImage
import com.gu.mediaservice.lib.logging.{LogMarker, MarkerMap}
import com.gu.mediaservice.model.{Bounds, Dimensions, ImageMetadata, Instance, Jpeg, Png, Tiff}
import org.scalatest.time.{Millis, Span}
import com.gu.mediaservice.model.{Dimensions, Instance, Tiff}
import com.gu.mediaservice.model._
import org.scalatest.concurrent.ScalaFutures
import org.scalatest.funspec.AnyFunSpec
import org.scalatest.matchers.should.Matchers
import org.scalatest.time.{Millis, Span}

import java.io.File
import java.lang.foreign.Arena
import scala.concurrent.ExecutionContext.Implicits.global
import scala.concurrent.{Await, Future}
import scala.concurrent.duration.{Duration, SECONDS}

// This test is disabled for now as it doesn't run on our CI environment, because GraphicsMagick is not present...
class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
Expand All @@ -25,6 +22,12 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
implicit override val patienceConfig: PatienceConfig = PatienceConfig(timeout = Span(1000, Millis), interval = Span(25, Millis))
implicit val logMarker: LogMarker = MarkerMap()

private val metadata = ImageMetadata(
credit = Some("Tony McCrae"),
copyright = Some("Eel Pie Consulting Ltd"),
suppliersReference = Some("eelpie-123")
)

describe("thumbnail") {
it("should write thumbnail to output file") {
val image = fileAt("IMG_4403.jpg")
Expand Down Expand Up @@ -87,6 +90,118 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

describe("resize") {
it("should output resized image to file in chosen format") {
implicit val arena: Arena = Arena.ofShared()
val fullSizedImage = VImage.newFromFile(arena, fileAt("IMG_4403.jpg").getAbsolutePath)
val imageOperations = new ImageOperations("")

val outputFile = new File("/Users/tony/Desktop/out5.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(1000, 800), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly in sRGB") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out6.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB colour spaces correctly as PNG") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out7.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB 16 bit colour spaces correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val fullSizedImage = VImage.newFromFile(arena, fileAt("halfdome_LAB16.tif").getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out8.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render PNG with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("with-alpha.png")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/resized-png-with-alpha.png")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Png)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}

it("render LAB TIFF with alpha correctly") {
implicit val arena: Arena = Arena.ofShared
val imageOperations = new ImageOperations("")

val image = fileAt("lab8-with-alpha.tif")
val fullSizedImage = VImage.newFromFile(arena, image.getAbsolutePath)
val outputFile = new File("/Users/tony/Desktop/out13.jpg")

val eventuallyResized = imageOperations.resizeImageVips(fullSizedImage, Dimensions(800, 600), 95, outputFile, Jpeg)

whenReady(eventuallyResized) { resized =>
arena.close()
resized.isFile should be(true)
}
}
}

describe("alpha") {
it("should return false for RGB for a Jpeg with no alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("rgb-wo-profile.jpg").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(false)
}

it("should return true for PNG with alpha") {
implicit val arena: Arena = Arena.ofShared
val image = VImage.newFromFile(arena, fileAt("with-alpha.png").getAbsolutePath)
val hasAlpha = ImageOperations.hasAlpha(image)
arena.close()
hasAlpha should be(true)
}
}

describe("identifyColourModel") {
it("should return RGB for a JPG image with RGB image data and no embedded profile") {
val image = fileAt("rgb-wo-profile.jpg")
Expand Down Expand Up @@ -218,7 +333,91 @@ class ImageOperationsTest extends AnyFunSpec with Matchers with ScalaFutures {
}
}

// TODO: test cropImage and its conversions
describe("graphic detection") {
it("should return not graphic for true colour jpeg") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("exif-orientated-no-rotation.jpg").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(false)
arena.close()
}

it("should return is graphic for depth 2 tiff") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("flower.tif").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 4 png with alpha") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/tbbn0g04.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

it("should return is graphic for depth 8 indexed png") {
val arena = Arena.ofConfined
val image = VImage.newFromFile(arena, fileAt("schaik.com_pngsuite/basn3p08.png").getAbsolutePath)
ImageOperations.isGraphicVips(image)(arena) should be(true)
arena.close()
}

}

describe("cropping") {
val operations = new ImageOperations("")

it("should create unscaled master crop to resize from full sized images") {
implicit val arena: Arena = Arena.ofConfined
//val fullsizedImage = fileAt("Lab 16bpc (7d0b7c7b8e890d7e5d369093aa437bd833e20f71).tiff")
val fullsizedImage = fileAt("IMG_4403.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))
arena.close()
}

it("should create unscaled master crop from CMYK full sized image") {
implicit val arena: Arena = Arena.ofConfined
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 2000, 2400), metadata, None)

val outputFile = new File("/Users/tony/Desktop/master-from-cmyk.jpg")
operations.saveImageToFile(masterCrop, Jpeg, 95, outputFile, keep = Some(VipsRaw.VIPS_FOREIGN_KEEP_XMP))

arena.close()
}

it("should create files foreach crop size") {
implicit val arena: Arena = Arena.ofShared()
val fullsizedImage = fileAt("CMYK-with-profile.jpg")
val metadata = ImageMetadata()

val masterCrop = operations.cropImageVips(fullsizedImage, Bounds(100, 100, 3000, 2000), metadata, None)
val landscapeCropSizingWidths = Seq(
Dimensions(140, 100),
Dimensions(320, 200),
Dimensions(800, 600),
Dimensions(1000, 1200),
Dimensions(2000, 3000),
)
implicit val i: Instance = Instance("id")

val crops = operations.createCrops(masterCrop, landscapeCropSizingWidths.toList, "test-image-id",
Bounds(0, 0, 1000, 1200),
Jpeg,
new File("/Users/tony/tmp/crops"),
75
)

arena.close()
}
}

def fileAt(resourcePath: String): File = {
new File(getClass.getResource(s"/$resourcePath").toURI)
Expand Down
2 changes: 0 additions & 2 deletions container-images/jdk-vips/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,6 @@ RUN rm /tmp/vips-8.18.2.tar.xz
RUN rm -r /tmp/vips-8.18.2/

RUN apt -y --no-install-suggests install \
pngquant \
libimage-exiftool-perl \
libjemalloc-dev \
graphicsmagick \
graphicsmagick-imagemagick-compat
Loading
Loading