Source: Gemini Image Understanding Docs
Google Gemini API documentation on image input: inline vs Files API, 258-token tiles, media_resolution, normalized bounding boxes and segmentation.
Contents
- What: official API documentation (Image understanding)
- Author/publisher: Google
- Date: living document; read 2026-11-14
- Original: Image understanding
Summary
Covers how to pass images to Gemini, how they are tokenized, and Gemini’s native object detection and segmentation outputs.
Key claims
- Inline base64 requests are capped at 20 MB total (prompt + bytes); use the Files API for larger or reused images. Up to 3,600 images per request.
- Formats: PNG, JPEG, WEBP, HEIC, HEIF.
- Tokens: an image with both sides at most 384 px costs 258 tokens; larger images are tiled into 768x768 tiles at 258 tokens each.
media_resolution(introduced with Gemini 3) caps tokens per image or video frame: higher means better detail, more tokens and latency.- Object detection returns
box_2das[ymin, xmin, ymax, xmax]normalized to 0-1000; you rescale to original pixels. - Segmentation (from Gemini 2.5) returns box, label and a mask per object.
- Best practices: correct rotation, unblurred images, and with a single image put the text prompt before it in the input.