Agentic Vision: Google Introduces Breakthrough Capability in Gemini Flash 3

Google has unveiled a groundbreaking feature called Agentic Vision for Gemini Flash 3 on 28 January, which transforms image processing from a static observation into an active investigative approach. According to a blog post from Google, this new methodology merges visual reasoning with automated code execution to assess visuals in what it describes as a “Think, Act, Observe” cycle.

Enhanced Accuracy with Agentic Vision

Google asserts that this innovative approach will minimise hallucinations and enhance the accuracy of responses to visual tasks. The blog detailed how the model creates plans to zoom in, inspect, and manipulate images in a sequential manner, grounding responses in visual evidence.

Real-Time Image Annotation Capabilities

Reportedly, Agentic Vision allows for real-time image annotation. Instead of merely describing a scene, the model functions as an agent, executing Python code to showcase its findings. This method replaces vague probabilities with verifiable, code-driven actions, boasting a potential quality increase of 5-10%.

Addressing Challenges of Traditional Models

Google mentioned that standard LLMs often experience hallucinations during multi-step visual arithmetic. The Gemini 3 Flash circumvents this issue by shifting computation to a deterministic Python environment. The company is transitioning from models that simply “observe” to those that actively “investigate.”

Real-World Applications of Agentic Vision

The company provided several real-world applications, highlighting that “PlanCheckSolver.com, an AI-driven platform for building plan validation, enhanced its accuracy by 5% through the ability to execute code with Gemini 3 Flash to methodically inspect high-resolution inputs.”

Counting Digits with Visual Precision

In another instance, the model is tasked with counting the digits on a hand through the Gemini app. To eliminate counting inaccuracies, it employs Python to draw bounding boxes and numerical labels on each finger detected.

Availability and Future Updates

The Agentic Vision feature is currently accessible to developers through the Gemini API within the Google AI Studio development tool and Vertex AI in the Gemini app.

Furthermore, Google has outlined plans for upcoming enhancements to Agentic Vision, including expanding its capabilities to enable automatic decisions for when to rotate, zoom, or perform visual arithmetic without additional prompts.

The tech giant is also aiming to equip Gemini models with additional tools such as web and reverse image search functionalities. Lastly, there are intentions to broaden Agentic Vision to encompass larger, more powerful models beyond Flash.