AI News
All stories
Official

Gemini API lets tools return audio, video and Office files to the model

Google says function responses in the Gemini API can now carry audio, video and Office files, and can reference Files API URIs.

Products · · Updated · 1 min read

Google said on November 5, 2026 that tools calling into the Gemini API can now send audio, video and Office files back to the model, not just images, PDFs and plain text. The change applies to function responses, the messages a developer's code returns after the model asks a tool to run. Google also added a way to return a Files API URI in place of the file's bytes. The same release note lists a displayName field on inlineData and fileData.

On the GenerateContent API, the new file types appear as parts of the function response. The Files API option is a fileData part that points at an uploaded file rather than embedding it in the request. The release note says video parts also accept a mediaProcessing setting with two values, STATIC or AGENTIC. It doesn't say what either mode does to the video or how it changes what the model sees.

The Interactions API gets a parallel change. Its function_result steps now accept document, audio and video content blocks, alongside the existing text and image blocks, and each can carry data inline or as a URI. Video blocks are available in v1beta and v1alpha and accept a processing option, which the note doesn't describe. The release note doesn't give size limits, supported file lengths or pricing for function-response media, so the scope of the change is clear only at the level of which types and which fields are accepted.

What changed

Function responses in the Gemini API can now carry audio, video and Office files, and can reference a Files API URI in place of the file's bytes.

Type: New capability · Applies to: GenerateContent API, Interactions API, v1beta, v1alpha

Why it matters

Agents can now receive richer media from tools directly, which removes some workarounds for multimodal tool pipelines. Developers on the beta endpoints should check which fields are available in their API version.

What you can do

If your agents pass media through tool calls, consider switching to Files API URIs and confirm the API version you use supports the new fields.

How the story developed

  1. The changelog adds the fileData URI option, displayName and mediaProcessing settings, and the matching Interactions API content blocks.

Original source

Where we saw it

1 source

Gemini API changelog

AI wrote this summary from the sources listed, and nobody reviewed it individually before publication. Check the original source before you rely on it. How it works