Gemini development for photo, video and document features

Google's Gemini models read text, images, video, audio and PDFs in one request. You reach them through the Gemini API with an API key, or through Vertex AI on Google Cloud. We build them into web and mobile products, with the parts around the model: file uploads, schemas, function calls, safety settings and a review step where an answer matters.
Google Gemini
  • Photo, video, audio and PDF understanding
  • Gemini API or Vertex AI, chosen for your needs
  • JSON output and function calls your code checks
  • Answers grounded in Google Search when you need current facts

When Gemini is the right choice

Gemini suits products built around media your users send:

  • Your users upload photos or video and you need the model to describe or measure what it sees. Gemini processes both the picture and the sound in a video.
  • You need to transcribe or summarize audio, tell speakers apart or refer to a moment by its timestamp.
  • You need to read long material in one request. Google says many of its models have context windows of a million tokens or more.
  • You already run on Google Cloud and need IAM, enterprise terms and support. Vertex AI gives you those with the same SDK.

When Gemini is not the right choice

Plan around these before you commit:

  • You plan to send customer data on the free tier. Google's terms say it may use that content to improve its products and human reviewers may read it, so use a paid tier or Vertex AI.
  • You need to keep uploaded files in Google's storage. Files sent through the Files API are deleted after 48 hours, so keep your own copy.
  • You need to cache or resell search-grounded answers. Google's terms require showing the Search Suggestions with each grounded answer and forbid caching or syndicating them.
  • You need fine detail from fast motion in video. The default sampling is one frame per second, so raise the frame rate or send stills.

What we build with Gemini

Typical projects, described in general terms.

Photo inspection

Customer or field photos read for type, color, condition and visible damage, with a person reviewing the result.

Estimates from photos and video

Items and approximate size worked out from uploads, then confirmed with the customer in the conversation before a quote.

Chat assistants

Assistants that answer from your data, call your functions and can switch between model providers behind one back end.

Document reading

PDFs read with their charts and tables, and fields extracted to JSON.

Audio and call notes

Transcripts, summaries and speaker labels from recordings, with timestamps for each section.

Answers with current facts

Grounding with Google Search for questions about recent events, shown with the sources and suggestions Google requires.

Gemini API or Vertex AI

Google offers Gemini two ways. The Gemini API uses an API key from Google AI Studio, and Google says most developers should use it unless they need specific enterprise controls. Vertex AI, which Google's documentation now calls Gemini Enterprise Agent Platform, runs in your Google Cloud project. It uses service accounts and IAM, and adds enterprise terms for data processing, support with SLAs, compliance certifications such as HIPAA and SOC 2, and Provisioned Throughput for dedicated capacity. Both use the same Google Gen AI SDK; the client is set up with a project and location for Vertex AI. Supported regions may differ between the two, and models tuned in AI Studio must be retrained on the platform. We choose by who owns the data, where it must stay and what support you need.

What a Gemini integration needs beyond the API

  • File uploads: large photos, videos and audio go through the Files API rather than inline in the request. Files are deleted after 48 hours and cannot be downloaded again, so your system keeps the original.
  • Media settings: video is sampled at one frame per second by default, with clipping and a custom frame rate available. Timestamps use the MM:SS format. Object detection returns boxes on a 0 to 1000 scale, which we convert back to the original image.
  • Schemas: structured output supports a subset of JSON Schema. Some keywords, such as string length limits, are silently dropped, and very large schemas can be rejected. Google says to validate values in your application, so we do.
  • Function calls: the model chooses the function and its arguments but does not run it. We validate each call before running it and keep the active tool set small, as Google recommends.
  • Safety settings: you can set thresholds for harassment, hate speech, sexually explicit content, dangerous content and jailbreak attempts. Protections against core harms, such as child safety, cannot be turned off. Blocked content returns an error, so the product needs a message for that case.
  • Long prompts: Google advises putting the question at the end of a long prompt, and notes accuracy drops when the model must find many separate facts at once. Context caching cuts the cost of reusing the same material.

How we work

  1. Choose the access route: the Gemini API or Vertex AI, decided by data terms, region and support needs
  2. Test with your own media: real photos, videos and documents, including poor and unusual ones, before the prompt is fixed
  3. Add checks around the model: schema validation, function-call checks, safety settings and a person in the loop where a result matters
  4. Hand over properly: prompts, upload flow, safety settings and cost monitoring documented, with notes on reading a failed request

Articles

Guides from our team on the questions buyers ask about this service.

All articles

Related technologies

Technologies that appear alongside Gemini in our case studies. Select one to see our work with it.

Frequently asked questions

Should we use the Gemini API or Vertex AI?

Google recommends the Gemini API for most developers. Vertex AI suits teams that need Google Cloud IAM, enterprise data terms, SLAs, compliance certifications or dedicated capacity. Both use the same SDK, but supported regions can differ.

Can Gemini understand photos and video our customers upload?

Yes. Gemini reads images, video and audio. Large files go through the Files API, which keeps them for 48 hours. Video is sampled at one frame per second by default, and both its picture and sound are processed.

Can Gemini return JSON our code can rely on?

Gemini can be given a JSON schema and returns syntactically valid JSON, but Google says to validate the values in your application. Only a subset of JSON Schema is supported, and some keywords are dropped without an error.

Can Gemini answer with current information from the web?

Yes, with grounding with Google Search. The model decides when to search and returns the queries, source links and Search Suggestions. Google's terms require showing the suggestions with the answer and forbid caching or reselling grounded results.

How can we work together?

There are three ways, on the same terms:

  • Dedicated team, best for ongoing work: interview the developers, then they work on your project full or part time.
  • Fixed scope, best when requirements are clear: we agree the scope and the price up front, and deliver against it.
  • Time and material, best when the scope will change: you pay for the time used and can change priorities as the work goes on.

Whichever you choose, we sign an NDA before work starts, you own the code we write, and every change is reviewed in your repository.

Can I interview the developers before we start?

Yes. For a dedicated team you meet the developers in an online call and interview them before they join your project.

Building a feature on Gemini?

Tell us what your users will send, where the data must stay and what the answer is used for. We will say which route fits and what the integration involves.

Tell us about your project

Tell us what you need built, fixed or improved. We come back with questions, a suggested approach and an estimate.