Image service helper.

[stephenwf]

A new generic helper for image services and canvases to resolve real, working thumbnails at a particular size, with fallbacks.

Some notes:

const thumbnail = imageService(service)
  .try({ width: 512, height: 512 })
  .try({ width: 256, height: 256 })
  .try('default')
  .asyncTry({ width: 512, height: 512 })
  .asyncFallback({ width: 64, height: 64, unsafe: true })
  .default('https://example.org/placeholder.jpg')
;

thumbnail.image; // The image.
thumbnail.promise; // The async version.
thumbnail.service; // Image service
thumbnail.asyncService; // Promise of the full image service.


const annotation = imageService(service)
  .crop({ x: 100, h: 100, width: 240, height: 240 })
  .try({ width: 256, height: 256 })
  .try({ width: 512, height: 512 })
  .default(thumbnail)
;

annotation.image; // The image.
annotation.test(); // HEAD on the image, if fails, `image` updates

annotation.store; // Reactive store, for updates.

// Signals.
const thumbnail imageService(
  service, { signal: controller.signal }
);

// Chaining?
const firstThumb = thumbnail.try({ ... });
if (!firstThumb.image) {
  const secondThumb = firstThumb.try({ ... });
}

// Hook?
const thumbnail = useImageService(service, init => 
  init.try({ ... }).default('...')
);

// Canvas finding.
const canvasThumbnail = canvasImage(manifest.items[0])
  .try({ width: 512, height: 512, from: 'thumbnail' })
  .try({ width: 512, height: 512, from: 'items' })
  .try({ width: 256, height: 256, from: 'thumbnail' })
  .try({ width: 256, height: 256, from: 'items' })
;

Ability to generate "virtual" images, for example from level0 tiles.

const fullImage = imageService(service).full({ stitch: true });

Which would be compatible with cropping:

const fullImage = imageService(service)
  .crop({ x: 100, h: 100, width: 240, height: 240 })
  .try({ width: 1024, height: 1024, stitch: false })     // <- (1)
  .asyncTry({ width: 1024, height: 1024, stitch: true }) // <- (2)
;
// Run this once, to check if (1) resolved, which will fallback to loading (2).
fullImage.test();

// Or manually:
fullImage.fail(fullImage.image);

Can pass a function to try, for custom logic. Should return {url, height, width}

const fullImage = imageService(service)
  .try(service => generateMySize(service))
  .try(service => ({ url: 'https://example.org/fallback', height: 256, width: 256 }))
;

Draft of try options:

interface TryOptions {
  // Exact height OR width, matches `!w,h` in the IIIF Image API specification:
  // The extracted region is scaled so that the width and height of the returned image are not greater than w and h, while maintaining the aspect ratio. The returned image must be as large as possible but not larger than the extracted region, w or h, or server-imposed limits.
  width: number;
  height: number;

  // Max width/height
  maxWidth: number;
  maxHeight: number;

  // Min width/height - will change the range of sizes.
  minWidth: number;
  minHeight: number;

  // With the min/max options, which should be prioritised.
  // "largest-size": will choose a larger image IF it's in the sizes array
  // "smallest-size": will choose a smaller image IF it's in the sizes array
  // "closest-size": will choose the closest size (area) IF it's in the sizes array
  // "largest-possible": will prefer larger images, even if not in the sizes array
  // "smallest-possible": will prefer smaller images, even if not in the sizes array
  sizePriority: 'largest-size' | 'closest-size' | 'smallest-size' | 'largest-possible' | 'smallest-possible' | 'exact';

  // Optional max area, useful if known - must be less than the maxArea specified by the loaded image service.
  maxArea: number;

  // The original width/height of the resource - useful for service that don't have width/height - this could be enough to make Image requests.
  resource: { width?: number; height?: number }
  
  // Optional format, quality and rotation to try.
  format: string;
  quality: string;
  rotation: number;

  // Cropped region to select - short hands available.
  region: {x: number; y: number; width: number; height: number};

  // If the image request should be made with cors, and optional headers - for auth.
  cors: boolean;
  headers: Record<string, string>;

  // Wether a virtual image should be created by requesting tiles (e.g. level0 services)
  stitch: boolean;

  // Profile - only matches if the profile matches.
  profile: 'level0' | 'level1' | 'level2';

  // Use full/full tile, if available - useful for level0 services
  fullTile: boolean;
};

Size priority.

Still need a bit more thought. When you request a size:

try({ width: 256, height: 256 });

There will only be one possible output, from the image service. !256,256 or eqv.

However, if you request a range of sizes:

try({ minWidth: 200, maxWidth: 1024, maxHeight: 1024 });

Then it could look for tiles and sizes that are suitable first. So if you added: largest-size

try({ minWidth: 200, maxWidth: 1024, maxHeight: 1024, sizePriority: 'largest-size' });

And the sizes array was:

{
  "sizes": [
    {"width":73,"height":100},
    {"width":145,"height":200},
    {"width":290,"height":400},
    {"width":725,"height":1000}
  ]
}

Then it would return /725,1000/ image request. However, if you set it to smallest-size then
{"width":290,"height":400} would be returned.

closest-size only works if you also provide a height and width:

try({ 
  width: 200, 
  height: 200, 
  maxWidth: 1024, 
  maxHeight: 1024, 
  sizePriority: 'closest-size' 
});

This would return {"width":145,"height":200}. The "area" is the closest value.

• 200x200 - 40,000 <- target
• 73x100 - 7,300
• 145x200 - 29,000 <- closest
• 290x400 - 116,000
• 724x1000 - 724,000

The options: smallest-possible and largest-possible work the same as largest-size and smallest-size for level0 image services, but for level1 and level2 they will return either the max or minimum size specified with a !w,h request.

[rsimon]

Just to say that I'd love to see this as a feature!

Is there already code in the works? FWIW: for our IMMARKUS annotation platform, I have implemented a Web Worker crop utility that I'm using to crop snippets from images stored locally, or fetched from IIIF Simple Image-type manifests (provided that CORS is enabled on the server).

I have not yet attempted to do stitching of Level 0 tiles, but this may become a requirement for IMMARKUS eventually. In case it's useful:

• getImageSnippet
• getImageSnippetWorker

[stephenwf]

Oh that nice! So the web worker serves them as if they were images. Sounds similar to the stitching, perhaps those use-cases could be grouped together into a library on top. Really interesting approach using the web worker.

[rsimon]

Exactly, the web worker serves the result as a data array, and that can then be displayed as a data URL, roughly like so:

<img src={URL.createObjectURL(new Blob([snippet.data]))} />