10 questions every startup should answer before moving to production with their AI prototype

It’s never been easier to start an AI-powered startup on Google Cloud. 

You grab an API key from Google AI Studio at breakfast, paste it into Antigravity, and by lunch you’ll have a nascent prototype of your product.

But it’s not all one straight line to progress. It’s common to bump into these three challenges as you build out your stack:

• A leaked API key racks up a large bill in 48 hours.

• A “quick” migration from AI Studio to Gemini Enterprise Agent Platform stalls the roadmap for weeks because nobody on the team owns Identity and Access Management (IAM).

• The launch works, until the app starts returning HTTP 429 Too Many Requests because of default per-project quotas, and there’s no clean path to more capacity without paying a premium.

None of these are unique edge cases. . They’re  default failure modes of moving fast without a plan, and we’ve all done it at least once.

Below are the 10 questions every startup should be ready to answer before they scale,  grouped into the three phases where decisions can shape your future: 

1. Onboard (setting up your own projects and identities right)

2. Scale (getting more throughput without breaking the bank) 

3. Govern (keeping costs, keys, and agents from running away). Each question ends with a short, runnable snippet you can copy into your own project today.

These ten are scoped to the prototype-to-production transition itself. Adjacent decisions that matter just as much but aren’t specific to that move, your data layer and RAG architecture, CI/CD, network design, are deliberately out of frame here. 

Onboard: get the foundation right (in the first hour).

#1 Where should I start: Google AI Studio or Gemini Enterprise Agent Platform?

Both surfaces expose the same Gemini family of models, but they solve different problems.

• Google AI Studio (with the Gemini Developer API) is the fastest path from an idea to working code. A browser IDE, an API key, a generous free tier, and no cloud project to configure. It’s where most ideas should start, and Google’s own guidance says as much.

• Gemini Enterprise Agent Platform (formerly Vertex AI) has the same Gemini models (plus 3rd party and OSS ones)  with enterprise controls around them: IAM and service-account auth instead of raw keys, VPC Service Controls, Cloud Logging and Monitoring, reserved capacity, regional endpoints, and the compliance surface your first enterprise customer’s security review will ask about.

The right answer for most startups is both, sequenced deliberately: first prototype in AI Studio, then migrate before you have real users. The danger for startups is treating them as interchangeable solutions, AI Studio’s simple key model does not translate to enterprise controls, and Agent Platform’s IAM model might look like overkill until the day it saves you from a stolen-credential incident.

It’s less work than it sounds like.

The unified google-genai SDK targets both:

The rule of thumb: the day you have users who are not you or your startup colleagues, you should already be on Gemini Enterprise Agent Platform.

#2  How do I set up a Google Cloud project without becoming an IAM expert?

The biggest reason startups stall on the migration to Agent Platform isn’t the code, it’s the operational leap from “here’s an API key” to a cloud project with folders, service accounts, org policies, logging, and IAM bindings. If your team doesn’t have a dedicated cloud admin, that first project setup can eat a week of engineering time. 

Three moves cut that dramatically:

1. Use an opinionated project template instead of clicking through the console. The Cloud Setup checklist and the Google Cloud Architecture Framework give you a production-grade folder hierarchy (prod / non-prod / dev), a central logging + monitoring project, Security Command Center turned on, and baseline org policies, without you having to design them from scratch.

2. Enable the APIs you’ll actually use, once. Batch it so you’re not doing it project-by-project when you need it. The billing-link step is not optional. Every paid API you’re about to enable will refuse to activate on a project with no billing account attached, so we handle that first.

3. Let Gemini pick the roles, but ask it for the narrow ones. You don’t have to memorize the roles reference. In the Grant access dialog, Help me choose roles lets you describe the task in plain language, “this service account needs to call Gemini models and read one Cloud Storage bucket”, and get predefined roles back with the reasoning shown. One catch worth knowing on day one: by default it suggests roles that cover common journeys, which usually means a service’s Admin, Editor, or Viewer. Those are broader than you want. Say “least privileged” or “narrowest access” in the prompt and it returns granular roles instead. Same amount of typing, considerably smaller blast radius when a credential leaks.

   Sources: Get predefined role suggestions with Gemini assistance

Sources: gcloud services enable reference, · gcloud billing projects link (GA),  GE Agent Platform environment setup.

If you’re a solo founder, resist the urge to build in your personal GCP account. Create a proper organization or self-owned org first, then create the project inside it. That single decision can make everything else, fromIAM to billing and audit, dramatically easier.

#3 I’m on Google Cloud, how should my code actually authenticate: API keys, service accounts, or user credentials?

There’s a hierarchy of safety here, and the easiest option is rarely the right one in production.

• Raw API keys are fine for local prototyping. They are dangerous in production because they are long-lived, easy to leak into a client bundle or a public repo, and grant unbounded access until you notice.

• User credentials via OAuth (application default credentials) are best for interactive tools, CLIs, and any code that runs on a developer’s laptop.

• Service accounts with least-privilege IAM roles are the right answer for anything running on a server, in a container, or in a scheduled job.

The pattern you’re aiming for is one where your code never sees a key at all. It just calls the Google Auth library, which quietly reads Application Default Credentials (ADC) from the environment,  a short-lived token minted for whichever service account is attached to your Cloud Run service, GKE workload, or Compute Engine VM. You get enterprise-grade auth without writing any auth code.

Do one last favor to your future self: give that service account the minimum IAM role your workload actually needs,  usually roles/aiplatform.user for calling models, not the broader admin roles. It takes an extra 30 seconds and prevents the credential from becoming a master key if it leaks.

#4 When should I actually stop procrastinating and migrate from AI Studio’s API key to Agent Platform’s IAM model?

Sooner than you’d like,  and the correct trigger is not when it breaks. It’s when any of these is true:

• Your key has left your laptop (checked into a repo, pasted into a Slack, shipped in a mobile app).

• You have more than one person on the team who needs to call the API.

• You’re spending more than a few hundred dollars a month.

• You’re about to onboard paying customers.

A potential pitfall that can catch growing startups off guard is simple: a leaked Gemini API key on an account that normally spends $180 a month gets scraped from a public repo and used to run distillation attacks,  accumulating tens of thousands of dollars in charges before the owner even sees the first billing alert. The Google Cloud Shared Responsibility Model is unambiguous: the customer is liable for charges incurred with their own valid credentials.

The migration itself is genuinely smaller than the anxiety around it. In google-genai it’s the two-line change shown in #1. What takes real time is the project setup around it, which is exactly why #2 exists.

Practical checklist for cutover day:

If step 3 prints a response, you’re on Agent Platform.

Phase B, Scale: get more capacity without paying a premium.

#5 Now that I’m shipping, why on earth am I getting all these HTTP 429 errors, and how do I make them stop?

429 Too Many Requests from Agent Platform almost always means one of two things:

1. You’ve hit the Dynamic Shared Quota (DSQ) ceiling for your project’s tier. DSQ is a shared pool sized against your project’s history,  new projects start with modest limits by design, to prevent abuse across the platform.

2. You’re calling a global endpoint during a global demand spike, competing with worldwide traffic for shared capacity.

The instinctive reaction is to file a quota-increase ticket. You can do that if you must,  but two architectural moves usually solve the problem faster and cheaper.

Pin to a regional endpoint. Over half of startup traffic on Agent Platform defaults to global routing. Pinning to a specific region (say us-central1) sidesteps global contention and typically improves latency at the same time. (One narrow exception, which we’ll get to in the next question: if you specifically want Priority PayGo, that feature currently only ships on the `global` endpoint. For everything else, pin regionally.):

Add real retry and backoff. A 429 is a retryable signal, not a fatal error. Any production client should have exponential backoff with jitter. The modern google-genai SDK ships this behavior built in, but only if you actually enable it. This is easy to overlook. Don’t reach for the classic `google.api_core.retry.if_transient_error` decorator you may have seen on older Vertex code. It’s designed for the legacy exception classes and does not recognize the new `google.genai.errors.APIError,  so it will silently pass 429s through without retrying. Use the SDK’s built-in retry options instead:

How do you see this coming?  Preferably not from a user telling you. Agent Platform publishes serving metrics to Cloud Monitoring, and there is a prebuilt dashboard you don’t have to assemble: Console → Agent Platform → Dashboard → Model observability. It gives you requests per second, token throughput, first-token latency, and error rates out of the box.

The metric to actually alert on is aiplatform.googleapis.com/publisher/online_serving/model_invocation_count. It carries an error_category label with values of user, system, or capacity. Alerting on capacity isolates genuine throttling from your own bad requests, which a raw 429 count won’t do.

One thing worth internalizing, because it trips people up: you cannot build a “warn me at 80% of my quota” alert for Standard PayGo. Under Dynamic Shared Quota there is no fixed per-project number to be at 80% of. A 429 means transient contention for shared capacity, not that you crossed a line. Percent-of-limit alerting only becomes meaningful once you’re on Provisioned Throughput, which does expose real limit metrics.

Sources: Agent Platform metrics list, Model observability dashboard, RetryOptions source,  core retry_base.py, genai errors.py,  reduce 429 errors, gcloud monitoring policies create, Dynamic Shared Quota.

Follow the Agent Platform rate limits documentation to understand what your project’s current ceiling actually is before you assume you’ve outgrown it. 

#6 Which consumption mode do I pay for: Standard PayGo, Priority PayGo, or Provisioned Throughput? 

Three consumption models, three completely different workload shapes, and three completely different ways to proceed. Picking the right one can help startups see meaningful savings on AI bills. First let’s define them and then see when they are, or aren’t, a good fit:

Standard PayGo (DSQ): Pay per token from a shared pool; cheap, no guarantees.
Priority PayGo: Pay per token at a premium to jump the queue.
Provisioned Throughput (PT): Prepay for reserved capacity; predictable, use it or lose it.

The dominant startup mistake is buying PT too early. Usually  this happens the  week after a big launch when it feels like traffic will only ever go up. PT is reserved capacity. You  pay whether you use it or not, and it only starts paying you back once your baseline is genuinely predictable, not just aspirational.

Here’s a pragmatic sequence:

1. Weeks one through four on Standard PayGo. Use it to measure your real request shape (tokens per minute at p50 and p99, request bursts, batchable vs. real-time split).

2. When you get your first bad 429 storm, flip on Priority PayGo for the traffic that actually matters. It’s a config change, not a purchase order,  nobody in procurement needs to be involved:

3. Once you can predict your baseline TPM, buy PT to cover the flat baseline and let anything above it overflow to PayGo. That’s the combined pattern Google recommends for exactly this reason. Best of both worlds, not marketing spin.

 Sources: Priority PayGo docs, google-genai HttpOptions source, GEAP REST reference. 

#7 Which of my requests actually need to be live, and which should be batch jobs?

Most startup workloads are secretly batch jobs pretending to be real-time. Every one you move off the interactive path frees up DSQ headroom for the traffic that genuinely needs to be fast,  the traffic where a user is actually watching a spinner.

Three questions to help you sort your traffic:

• Does a human have to see the result within a second? That means:  Live inference.

• Can the user wait a few seconds and see a spinner? That means:  Still live, but a candidate for streaming.

• Would the user tolerate “we’ll email you when it’s ready” or “check back in a bit”?  That means: Batch prediction.

Batch prediction on Agent Platform runs in a completely separate queue, does not consume your interactive DSQ, and is typically about half the price of on-demand inference. That’s a rare double win: faster live traffic and a lower bill.

Common candidates: nightly document summarization, background classification of new signups, bulk translation, embedding backfills, evaluation runs against your test set. If any of those are on your live path today, moving them is often the single highest-leverage change you can make this week.

Govern: Keep costs, keys, and agents under control.

#8 How do I set spend caps that actually reduce cost, and not just send me polite emails while my bill triples?

Until recently the honest answer was that budgets only notify, and you had to build your own brake pedal. That changed in July. There are now three mechanisms, and you should think of them as layers.

1. A spend cap budget (Preview). Cloud Billing budgets can now enforce rather than just email. Set a spend cap on a project and, when usage costs cross 100% of the budget, Google pauses the service until you manually lift it. Agent Platform is explicitly on the eligible list, alongside the Gemini API, Cloud Run, and Cloud Run functions. Alerts still fire at 50% and 80%, so the pause isn’t a surprise.

Three things to know before you rely on it:

• Each cap covers one project and one eligible service. It is not account-wide protection. If you want Agent Platform and Cloud Run both capped, that’s two caps. 

• Enforcement is not instant and is based on estimated costs. Overages past the cap are billed as normal, so set the number below your real ceiling. Lifting it is manual, and service resumption can take up to an hour. It also pauses Provisioned Throughput usage, so if you’ve prepaid for capacity, a cap hit stops that too.

• It’s in Preview as of publication, and the eligible-service list is documented as growing. Check the current list before you design around it.

2. A billing budget with a Pub/Sub trigger that disables billing. Still the right tool when you need blast radius the spend cap can’t give you: multiple services at once, an entire project, or a service that isn’t eligible yet. When the budget hits a threshold, Pub/Sub fires a Cloud Function that detaches the billing account, which stops all billable activity within minutes. Blunter and more dangerous than the native cap — it can leave resources unrecoverable — so reach for it second, not first. Full walkthrough: Automatically respond to budget notifications.

Sources:  Manage spend cap budgets, Set up programmatic notifications gcloud billing budgets create reference, Cloud Billing budgets concepts, Disable billing with notifications walkthrough, Programmatic notification payload schema.

Two things to get ahead of  for, as the defaults can cause unexpected issues: 

1. Limit your budget scope: Without –filter-projects, your budget applies to your entire billing account. A spike in any project will trigger the kill switch for everything. 

2. Deploy locally: The budget notification doesn’t specify which project is affected. To ensure the kill switch only affects the intended project, deploy your Cloud Function in the same project you’re protecting (e.g., my-startup-prod).

Then wire up a tiny Cloud Function to that topic that calls projects.updateBillingInfo to unlink the billing account when the 100% threshold fires. That is your circuit breaker.

Mechanical ceilings via quota overrides. Even if you never set up the above kill switch, you can cap the rate at which cost can accumulate by setting explicit per-model, per-region quotas below the platform default. If your app never legitimately needs more than 500 requests per minute for gemini-2.5-pro, cap it there in the Cloud Quotas console, a leaked key can’t burn what the quota flatly refuses to serve.

#9 Where should I actually keep secrets? (Not in .env files!)

The short answer is: Secret Manager. Not  in environment variables, not in .env files, and never in your repo. Grant read access via IAM only to the service account that needs it.

Then two little disciplines that pay for themselves the first time you need them:

• Rotation on a schedule and on suspicion. Secret Manager versions are cheap; treat them as immutable and roll forward. 

• Detection when a secret leaks. Secret Manager notifications and Google Cloud’s Sensitive Data Protection can catch keys checked into a repo or pasted into a log stream,  before an attacker does.

For any AI application that acts on a user’s behalf, calls Gmail on their behalf, reads a Drive folder, hits a third-party SaaS with the user’s credentials, do not store a long-lived token. Use OAuth 2.0 with short-lived access tokens and a refresh flow, so that when a user rage-quits or a compromised account gets revoked, the agent loses access at the same time. 

#10  How do I stop my brand new AI agent from doing something it absolutely shouldn’t?

An agent that can call tools, browse the web, or execute code needs the same defense-in-depth thinking as any other production service, arguably more, because it makes decisions that neither you nor the model can fully predict in advance.

Four layers, none optional once you have real users:

1. Identity for the agent itself. Give the agent its own service account, scoped only to the resources and tools it genuinely needs,  the exact same least-privilege principle as any other workload. Agent Engine supports first-class agent identity so every action can be attributed to a specific agent instance in your audit logs.

2. Sandboxed code execution. If your agent runs generated code,  a common pattern for data-analysis or “run this Python for me” flows, do not run it in your application process. Use an isolated sandbox so a bad combination can’t touch your production data.

3. Prompt and response filtering. Model Armor sits in front of your model calls and screens for prompt injection, jailbreaks, sensitive-data exfiltration, and off-brand output,  all of which are essentially guaranteed the moment you have real users being real users.

4. Behavioral monitoring. Security Command Center with threat detection flags anomalies in agent behavior,  a service account suddenly calling an API it’s never touched before, an agent reaching out to an unfamiliar external host, an unexpected spike in privileged operations. In near-real-time.

None of these are optional once your agent is acting on behalf of a real user or handling real money.

Your homework, so to speak:

1. Audit for raw API keys in your repo, your notebooks, and your production runtime. Rotate anything that shouldn’t be there.

2. Move any workload that doesn’t need a synchronous response to the Batch API.

3. Turn on the Model observability dashboard and put one alert on capacity errors, so the next 429 reaches you before it reaches a customer.

4. Set a spend cap on the project and, and keep an eye out for 50% and 80%alerts, if usage crosses 100% of the budget, Google will pause the service until you manually lift it.

Do those three things this week and you’re already ahead of most startups shipping AI features. 

Have a scenario you’d like us to cover next? Reach us at Google Cloud for Startups.