feat(image): prevent scanning oversized container images

[nikpivkin]

Description

This PR adds the --max-image-size flag. If the flag is specified and the uncompressed image size exceeds the maximum size, Trivy exits with an error. If the flag is not specified, the behaviour does not change.

For more implementation details, see #8176

Related issues

• Close feat: prevent scanning oversized container images #8176

Checklist

• I've read the guidelines for contributing to this repository.
• I've followed the conventions in the PR title.
• I've added tests that prove my fix is effective or that my feature works.
• I've updated the documentation with the relevant information (if needed).
• I've added usage information (if the PR introduces new options)
• I've included a "before" and "after" example to the description (if the PR is a user interface change).

[nikpivkin]

@knqyf263 I was thinking that we could download the compressed layer and then decompress it ourselves. I saw that downloader.Download is used for decompression. Does it support all the compression types that go-containerregistry does?

[knqyf263]

This is what was in my mind.

	layer, err := a.image.LayerByDiffID(h)
	if err != nil {...}

	rc, err := layer.Uncompressed()
	if err != nil {...}

	f, err := os.Open(diffID + ".tar")
	if err != nil{...}

	io.Copy(f, rc)

[nikpivkin]

@knqyf263 Then we can't display the progress bar correctly, because we don't know the size of the uncompressed layer.

[knqyf263]

Can't we use the progress in ggcr?
https://github.com/google/go-containerregistry/blob/6bce25ecf0297c1aa9072bc665b5cf58d53e1c54/pkg/v1/remote/progress.go#L25

[knqyf263]

If it doesn't work, we can use layer.Compressed, inject the progress bar and re-implement uncompressed logic.
https://github.com/google/go-containerregistry/blob/6bce25ecf0297c1aa9072bc665b5cf58d53e1c54/pkg/v1/partial/compressed.go#L50-L78

[nikpivkin]

@knqyf263 I kept the old behavior, if no flag is passed, to avoid unnecessary disk operations. But then the progress bar will be displayed only when the flag is passed. What do you think?

[knqyf263]

@nikpivkin I got an idea.
https://go.dev/play/p/fY8jM99don_r

[knqyf263]

This might be better.

// ProgressLayer wraps a v1.Layer to add progress bar functionality
type ProgressLayer struct {
	v1.Layer
}

func NewProgressLayer(layer v1.Layer) *ProgressLayer {
	return &ProgressLayer{ Layer: layer, }
}

func (l *ProgressLayer) Compressed() (io.ReadCloser, error) {
	size, err := l.Layer.Size()
	if err != nil {
		return nil, err
	}

	rc, err := l.Layer.Compressed()
	if err != nil {
		return nil, err
	}

	bar := pb.Full.Start64(size)
	bar.Set(pb.Bytes, true)

	return bar.NewProxyReader(rc), nil
}

[knqyf263]

Even better

// progressLayer wraps a v1.Layer to add progress bar functionality
type progressLayer struct {
	v1.Layer
}

func (l *progressLayer) Compressed() (io.ReadCloser, error) {
	rc, err := l.Layer.Compressed()
	if err != nil {
		return nil, err
	}

	size, err := l.Layer.Size()
	if err != nil {
		return nil, err
	}

	bar := pb.Full.Start64(size)
	bar.Set(pb.Bytes, true)

	return bar.NewProxyReader(rc), nil
}

func NewProgressLayer(layer v1.Layer) (v1.Layer, error) {
	return partial.CompressedToLayer(&progressLayer{ Layer: layer})
}

[nikpivkin]

@knqyf263 Ok, I'll think about how to use this with the progress bar pool otherwise there is flickering in the terminal when loading in parallel

[knqyf263]

@nikpivkin Actually, the progress bar should be discussed in #8186. I'll re-post it there.

[knqyf263]

BTW, Trivy also supports VM images. I'm unsure, but users may want to introduce the same functionality in the future. Do you think we want to rename --max-image-size to --max-target-size or something like that? Our VM scanning also targets VM "images". --max-image-size may be fine.

[DmitriyLewen]

I think that --max-target-size will confuse users, because they will try to use this flag in fs mode.

I think --max-image-size is good name.

Also i suggest to add this flag for VM scan in another PR.

[knqyf263]

I performed a quick test on Azure, which has a stable network. The current approach (without --max-image-size) seems faster. I'm sure we'll get different results depending on the environment, though.
https://gist.github.com/knqyf263/e0e4fa5bd1359fd9e68408f3c5244f1a?permalink_comment_id=5393170#gistcomment-5393170

[knqyf263]

We can delete the cache directory if checkImageSize returns an error, but how about other errors? If I understand correctly, Artifact.Clean() will not be called.

So, should we call the defer function before handling the error?

trivy/pkg/scanner/scan.go

Lines 160 to 165 in ec124bd

	defer func() {
	if err := s.artifact.Clean(artifactInfo); err != nil {
	log.Warn("Failed to clean the artifact",
	log.String("artifact", artifactInfo.Name), log.Err(err))
	}
	}()

[nikpivkin]

Clean takes artifactInfo, and if we call it before the error is checked, the artifactInfo will not be valid. Maybe we should clear the cache inside Inspect if any error occurred?

	defer func() {
		if err != nil {
			if rerr := os.RemoveAll(a.layerCacheDir); rerr != nil {
				log.Error("Failed to remove layer cache", log.Err(rerr))
			}
		}
	}()

[knqyf263]

I thought using Clean() is more straightforward, but you're actually right. We don't need this cache after Inspect is complete. It makes more sense to delete it in the method.