Add alt attributes to source elements

[johannesodland]

When picture elements are used for art direction (using source elements with media queries) they use distinct images depending on the environment/screen. These images might have a different layout to adapt to the screen, and would benefit from distinct replacement texts that match the selected image source.

Example

A picture element has been used for art direction. For vertical screens there is an image with a circle on the top and a box on the bottom. For horizontal screens there is an image with a circle on the left and a box on the right.

<picture>
  <source src="horizontal.jpg" media="(min-aspect: 1/1)">
  <img src="vertical.jpg" alt="A circle and a box">
</picture>

Proposal

source elements could support an alt attribute that is set on the img element when the source is selected. (Similar to how width and height properties now are set on img when a source is selceted: #5894)

<picture>
  <source src="horizontal.jpg" media="(min-aspect: 1/1)" alt="A circle to the left of a box">
  <img src="vertical.jpg" alt="A circle above a box">
</picture>

[annevk]

(Slightly related: #4478.)

[annevk]

cc @whatwg/a11y

[patrickhlauke]

it would make sense to me, yes (assuming it falls back on the alt for the <img> when there's nothing else specified in the <source>)

[scottaohara]

agreed, this proposal would be very helpful for instances where the different sources don't all match the base text alternative of the <img>.

And echoing Patrick, would want to make sure it's clear that authors only need to set an alt to a source when the base alternative text is not adequate. Otherwise, continue to use the alt from the fallback image for any source without an alt.

[hidde]

+1 to all, this would be a good addition and it closes a gap (if people were currently doing art direction that required different alternatives, they could not do that while enjoying the benefits of picture).

[aardrian]

Came here to say the same as @scottaohara and @hidde — this fits the art direction use case. So I also agree.

[zcorpan]

This was discussed in ResponsiveImagesCG/picture-element#254 back in 2014, then the conclusion was that the image ought to represent the same information even when doing art direction, and therefore alt should be the same.

[zcorpan]

@johannesodland do you have a real-world example where you think alt should be different between source elements?

[patrickhlauke]

randomly thinking: a <picture> for a site that shows a desktop/laptop when it's viewed on a large screen, a tablet on medium sized viewport, and a mobile/smartphone on small viewport ... maybe for a cutesy image of "our animated mascot reading our website" that tries to be "appropriate" to the viewport size (and inferred device) it's currently on. the alt would be changing for each image.

another one could be: where there's a large enough viewport, an image that shows a large overview of a location with a crowd, and has an inset (as part of the same image) of a zoomed-in section, to draw attention to that part. the alt would explain that the image shows the whole location, plus the inset. on smaller viewports it might just show the inset picture as a standalone image, and the alt would just refer to that.

[johannesodland]

@johannesodland do you have a real-world example where you think alt should be different between source elements?

One example I can remember is this article: https://www.nrk.no/ingvild-testet-sirkulaer-okonomi-i-praksis-1.15125257

For the header image of this article the journalists/photographer has chosen to use two different images of the same scene: For desktop there are three persons working, while on mobile there is a closer crop of the protagonist of the story.

When using "full-screen" images in articles, it is necessary to use different crops for different screens. This might leave out parts of the original image. I think the replacement text should describe what's visible in the image that is presented to the user. In this case the replacement text on desktop should contain information about the three persons, while on mobile it should only describe the main protagonist.

[aardrian]

@zcorpan

…the conclusion was that the image ought to represent the same information even when doing art direction…

In the nearly 7 years since then, and as predicted in the linked conversation, we see that real-world use has varied from the spec.

[johannesodland]

The real world users here are not only web authors, but end users of i.e. content management systems. These users are not developers, but journalists, designers, photographers, bloggers etc. I don't think there's any way we can ensure that every crop they decide to use of an image contain the exact same information.

With prefers-color-scheme designer might use different colors in illustrations to match the color scheme while still representing the same information. The alt text should be able to reflect the image source that is used.

[hidde]

@zcorpan said:

This was discussed in ResponsiveImagesCG/picture-element#254 back in 2014, then the conclusion was that the image ought to represent the same information even when doing art direction, and therefore alt should be the same.

Interesting! Makes sense to me… from what I've personally seen, I think in practice images will likely represent the same information in the art direction use case. And if the information really is so different that it warrants a different alt, an image may not be the best way to represent the information in the first place. In other words, the use case is probably not very common.

Having said that, I still feel being able to assign unique alts in the way proposed would be a useful markup option to have for an author. They can art direct today using two regular images with unique alts and some scripting, why not let them do the same with sources in a picture?

And, as @johannesodland says, authoring tools could let authors alts per source. Bit of a tangent, but I know in at least some CMSes alternative text is managed globally in some sort of media library, per file. That is usually not great as alts depend on page context, but may be just what is required in the art direction use case.

[johannesodland]

Bit of a tangent, but I know in at least some CMSes alternative text is managed globally in some sort of media library, per file.

Some of our media libraries indeed have with one alt per crop/file (simplified). As a result we either have to pick one of the alt texts, ask the editor/user to choose one or ask the them to enter a third alt text.

Having one alt per source would make it better and more intuitive for the editor user. (The crop shown to the end user will get the alt text that is stored together with that file/crop).

[johannesodland]

I work for a media organization, and a large part of our content are news related. The images are often more documentary and are seldom carefully planned and shot.

While most sources in the art directed pictures in essence contain the same information, cropping an image in multiple aspect ratios will always exclude some cursory information in one crop while keeping it in the other.

With only one alt text, some end user with screen readers will either get a text describing more than what is actually visible on the screen, or less.

Having one alt per source will improve the experience of an end user relying on alt text.

Having the exact same information in two images with different aspect ratios seems more like a technical purity than something that can be achieved in practice.

Having the option of using one alt per source would make it better for both end users (both editors and consumers) and web authors (designers and developers).

[zcorpan]

Thanks @johannesodland!

I would like to check HTTP Archive for pages that use data-alt attributes on source elements, if there's evidence of pages implementing the desired behavior with JavaScript today. (Unfortunately I couldn't do it today because there's some billing problem...)

[zcorpan]

OK I can run httparchive queries again.

Out of 6,239,076 pages, there are 35 unique pages using data-alt on source. (~0.00056% of pages)

https://docs.google.com/spreadsheets/d/1YKHe_je9If37X-4TE__h3LsQk3_4ICsDXISCRJ8w1Rw/edit?usp=sharing

It looks like most of these are using the exact same string in data-alt between source elements and the img's alt.

This doesn't show a lot of demand. (Are there any polyfills for this using a different attribute name?)

But maybe that's not so surprising. Maybe developers don't bother polyfilling to change the alt text based on which image is selected, but if HTML had this feature then CMSes would use it and users experiencing the alt text are presented more accurate information.

Queries

Getting all extracts of <picture>...</picture> instances:

SELECT
  *
FROM (
  SELECT
    page,
    url,
    ARRAY_TO_STRING(REGEXP_EXTRACT_ALL(LOWER(body), r'<picture(?:\s[^>]+)?>.+?</picture>'), '\n') AS markup
  FROM
    `httparchive.latest.response_bodies_desktop` )
WHERE
  markup != ""

Getting the instances with data-alt attribute on source a element:

SELECT
  page,
  url,
  markup
FROM
  `(temporary table)`
WHERE
  REGEXP_CONTAINS(markup, r"<source(?:[^>]+\s+)*data-alt\s*=")

[aardrian]

Would it be worth looking for any -alt value? IME, some frameworks do not use data- attributes, instead using their own prefixes. While data-alt feels like the best choice, I don't think it is necessarily the best signal on its own. Either way, it also only addresses cases where the value appears as an attribute on the node, not necessarily those where the value is in a JSON object.

[zcorpan]

@aardrian it's probably better to look at existing polyfills that implement this behavior (if there are any) and then search for pages using them.

[nhoizey]

It might be useful to sync this with the need to allow "empty" source as discussed in ResponsiveImagesCG/picture-element#243

If one of the source is empty, would the alt from the img be still relevant?

[r12a]

While i'm sympathetic to the goal here, introducing this information via attributes extends a problem we already have with alt (and title) vis a vis internationalisation: eg. you can't indicate that text in an attribute is in a different language from the element's textContent, and you can't indicate changes in language inside the attribute text that would allow for better assignment of fonts or styling. It also means that you can't use markup to indicate base direction.

I think the Internationalization WG will ask you to explore element-based solutions for this kind of thing when adding new features, rather than using attributes for natural language text, ie. something similar to the way that figcaption works with figure (in terms of the markup relationship – i know the semantic roles are different).

[Boldewyn]

That element-based approach was my initial try for a fix in ResponsiveImagesCG/picture-element#254 by re-using the for attribute on different figcaptions:

<figure>
  <picture>
    <source id="wide">
    <source id="narrow">
  </picture>
  <figcaption for="narrow">John Doe, left-most</figcaption>
  <figcaption><!-- default caption: -->John Doe, 2nd from left</figcaption>
</figure>

That seems to check all the “make easy things easy and hard things possible” boxes, but looks a bit clunky.