GCP Hosting for AI MVPs: Cloud Run, Vertex AI, and What to Skip

Start with Cloud Run

Cloud Run runs a container, scales to zero when idle, and scales up under load without you configuring anything. For an MVP that's almost always the right compute primitive on GCP.

It sidesteps the two usual failure modes: paying for idle VMs, and adopting Kubernetes before you have a problem Kubernetes solves.

Cold starts are the trade-off. A container that takes eight seconds to boot makes the first request after idle painful. Fix it by keeping the image small, doing lazy initialization, and setting a minimum instance count of one for anything user-facing.

When GKE is justified

Rarely, at MVP stage. Kubernetes earns its complexity when you're running many services with genuinely different scaling profiles, need fine-grained scheduling, or have a platform team to own it.

If your MVP is a web app, a worker, and a database, GKE is a distraction with an ongoing operational cost.

Vertex AI: worth it or not

Worth it when you're fine-tuning models, need a managed vector store via Vertex AI Search, or want Gemini with enterprise data-residency guarantees. The managed endpoints handle scaling and versioning properly.

Skip it if you're calling a model API and storing embeddings. At that stage Vertex adds a layer of abstraction and cost over what is fundamentally an HTTP call plus a database with pgvector.

Most AI MVPs we build fall in the second category and move to the first only when fine-tuning or residency becomes a real requirement.

The data layer

Cloud SQL for Postgres with the pgvector extension covers relational data and embeddings in one place. For MVP-scale corpora this is almost always sufficient and far simpler than a dedicated vector database.

Firestore if your data is genuinely document-shaped — but see the relational caveat that applies to any document store.

BigQuery for analytics, not for serving application queries. Teams sometimes reach for it as a general database and discover the latency and per-query cost model don't suit request paths.

What actually costs money

The surprises are consistent: idle GPU instances left running after an experiment, egress charges on data leaving the region, and Vertex endpoints provisioned with a minimum replica count and then forgotten.

Set a budget alert on day one. It takes two minutes and catches all three.

What we set up

Cloud Run for compute, Cloud SQL with pgvector for data, Secret Manager for credentials, Cloud Build for deploys, and budget alerts before anything else. Load testing against a realistic traffic shape so autoscaling settings are chosen rather than defaulted.