Google Vertex AI Setup

Back to README

Vertex AI is Google Cloud's enterprise platform for accessing Gemini models with VPC-SC, CMEK, IAM, regional/global endpoints, and the latest preview models. Use it when you need GCP-native auth, data residency, or features that aren't on the public Gemini API (Vertex-only previews, enterprise compliance).

Supported Models

Note: Preview and experimental models automatically use the v1beta1 API version. The Gemini 3.x preview models are global endpoint only — set GOOGLE_CLOUD_LOCATION=global.

Regional vs Global Endpoints

Vertex AI offers two endpoint types:

• Regional (e.g., us-central1, europe-west1): Low-latency, data residency guarantees

  https://us-central1-aiplatform.googleapis.com/v1/projects/{project}/locations/us-central1

• Global: Higher availability, required for Gemini 3.x preview models

  https://aiplatform.googleapis.com/v1beta1/projects/{project}/locations/global

The provider automatically selects the correct hostname and API version based on the region and model name. Set GOOGLE_CLOUD_LOCATION=global for Gemini 3.x preview models.

Step 1: Create a Service Account

export PROJECT_ID="your-project-id"

# Enable Vertex AI API
gcloud services enable aiplatform.googleapis.com --project=$PROJECT_ID

# Create service account
gcloud iam service-accounts create nous-vertex-ai \
  --display-name="Nous Vertex AI" \
  --project=$PROJECT_ID

# Grant the Vertex AI User role
gcloud projects add-iam-policy-binding $PROJECT_ID \
  --member="serviceAccount:nous-vertex-ai@${PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/aiplatform.user"

# Download the key file
gcloud iam service-accounts keys create /tmp/sa-key.json \
  --iam-account="nous-vertex-ai@${PROJECT_ID}.iam.gserviceaccount.com"

Step 2: Set Environment Variables

# Load the service account JSON into an env var (recommended — no file path dependency)
export GOOGLE_CREDENTIALS="$(cat /tmp/sa-key.json)"

# Required: your GCP project ID
export GOOGLE_CLOUD_PROJECT="your-project-id"

# Required for Gemini 3.x preview models (global endpoint only)
export GOOGLE_CLOUD_LOCATION="global"

# Or use a regional endpoint for stable models:
# export GOOGLE_CLOUD_LOCATION="us-central1"
# export GOOGLE_CLOUD_LOCATION="europe-west1"

Both GOOGLE_CLOUD_REGION and GOOGLE_CLOUD_LOCATION are supported (consistent with other Google Cloud libraries). GOOGLE_CLOUD_REGION takes precedence if both are set. Defaults to us-central1 if neither is set.

Step 3: Add Goth to Your Application

Goth handles OAuth2 token fetching and auto-refresh from the service account credentials.

# mix.exs
{:goth, "~> 1.4"}

# application.ex — start Goth in your supervision tree
credentials = System.get_env("GOOGLE_CREDENTIALS") |> JSON.decode!()

children = [
  {Goth, name: MyApp.Goth, source: {:service_account, credentials}}
]

Step 4: Configure and Use

# Option A: App config (recommended for production)
# config/config.exs
config :nous, :vertex_ai, goth: MyApp.Goth

# Then use it — Goth handles token refresh automatically:
agent = Nous.new("vertex_ai:gemini-3.1-pro-preview")
{:ok, result} = Nous.run(agent, "Hello from Vertex AI!")

# Option B: Per-model Goth (useful for multiple projects)
agent = Nous.new("vertex_ai:gemini-3-flash-preview",
  default_settings: %{goth: MyApp.Goth}
)

# Option C: Explicit base_url (for custom endpoint or specific region)
alias Nous.Providers.VertexAI

agent = Nous.new("vertex_ai:gemini-3.1-pro-preview",
  base_url: VertexAI.endpoint("my-project", "global", "gemini-3.1-pro-preview"),
  default_settings: %{goth: MyApp.Goth}
)

# Option D: Quick testing with gcloud CLI (no Goth needed)
# export VERTEX_AI_ACCESS_TOKEN="$(gcloud auth print-access-token)"
agent = Nous.new("vertex_ai:gemini-3.1-pro-preview")

Model settings

Everything below is passed as model settings — a plain map that Nous merges into the Vertex generateContent request body. Settings live at three layers, each merged over the previous one:

# Model layer — stored on the %Nous.Model{} struct behind the agent
agent = Nous.new("vertex_ai:gemini-2.5-pro", default_settings: %{temperature: 0.2})

# Agent layer — merged over the model layer
agent = Nous.new("vertex_ai:gemini-2.5-pro", model_settings: %{temperature: 0.2})

# Run layer — merged over the agent layer, for a single call
{:ok, result} = Nous.run(agent, "Summarize this.", model_settings: %{temperature: 0.9})

Any setting you leave out is omitted from the request entirely — Nous never substitutes a value of its own for the keys in this section, so the API's defaults apply.

This whole surface is shared between vertex_ai: and gemini:. Both providers build their request body through Nous.Messages.Gemini.build_request_params/3 (same wire format), so every key works identically against either prefix; whether a given value is accepted (a preview model, a Vertex-only native tool) is an API-side question, not a library one. The only Vertex-specific setting is :goth, and it is read from the model layer only — default_settings: %{goth: MyApp.Goth} or config :nous, :vertex_ai, goth: MyApp.Goth (see Step 4). Putting :goth in model_settings: has no effect.

The familiar cross-provider keys map here too: :temperature, :max_tokens → maxOutputTokens, :top_p → topP, :presence_penalty, :frequency_penalty, and :stop_sequences (or :stop) → stopSequences. Two escape hatches cover anything not listed: :generationConfig (a raw map, merged last into generationConfig) and :extra_body (merged into the top-level body, subject to the provider's blocked-key policy).

Thinking

:thinking_config maps to generationConfig.thinkingConfig for Gemini 2.5/3.x. Elixir keys (thinking_budget, include_thoughts) are camelized for you; native camelCase string keys are accepted verbatim, and any key Nous does not recognize is forwarded unchanged, so newer Vertex fields work without a library bump.

agent =
  Nous.new("vertex_ai:gemini-2.5-pro",
    model_settings: %{
      thinking_config: %{thinking_budget: 1024, include_thoughts: true}
    }
  )

{:ok, result} = Nous.run(agent, "Prove that the square root of 2 is irrational.")
result.output

With include_thoughts: true, thought summaries arrive as the assistant message's reasoning_content rather than in output. For tool-using thinking models, Vertex's thoughtSignature is preserved on each parsed tool call and echoed back on the next turn automatically — no configuration needed.

JSON output

Three settings feed Gemini's responseMimeType / responseSchema pair, in this priority order: :json_schema (a map) wins, then :response_format (%{type: :json_schema, schema: schema} or %{type: :json_object}, the cross-provider shape), then :json_response (a boolean, mime type only).

# Loose JSON: responseMimeType only, no schema enforcement
agent = Nous.new("gemini:gemini-2.5-flash", model_settings: %{json_response: true})

# Schema-constrained JSON: responseMimeType + responseSchema
schema = %{
  "type" => "object",
  "properties" => %{
    "city" => %{"type" => "string"},
    "population" => %{"type" => "integer"}
  },
  "required" => ["city", "population"]
}

agent = Nous.new("vertex_ai:gemini-2.5-flash", model_settings: %{json_schema: schema})
{:ok, result} = Nous.run(agent, "Which is the largest city in Japan, and how big is it?")
result.output

These are the raw provider knobs. If you want a validated Elixir struct back instead of a JSON string, use the agent-level output_type: option — see the Structured Output guide, which drives these same fields for you.

Safety settings

:safety_settings is a list of category/threshold maps placed at the top level of the request as safetySettings. Atom keys are stringified; entries written with string keys are passed through, as are keys Nous does not know about (e.g. "method").

agent =
  Nous.new("vertex_ai:gemini-2.5-flash",
    model_settings: %{
      safety_settings: [
        %{category: "HARM_CATEGORY_DANGEROUS_CONTENT", threshold: "BLOCK_ONLY_HIGH"},
        %{"category" => "HARM_CATEGORY_HARASSMENT", "threshold" => "BLOCK_NONE"}
      ]
    }
  )

{:ok, result} = Nous.run(agent, "Summarize this support ticket.")
result.output

Function calling, tool choice, and native tools

Function calling works on Vertex and Gemini. As of 0.16.0, tool declarations are serialized into Vertex's tools[].functionDeclarations format via Nous.ToolSchema.to_gemini/1 (which drops OpenAI's strict field and the schema keys Vertex rejects — additionalProperties and $schema, recursively). Notes claiming that tools are silently dropped for these providers describe pre-0.16.0 behaviour — pass tools: exactly as you would for any other provider.

:tool_choice accepts the friendly forms :auto, :any / :required, :none, and {:any, ["fn_a", "fn_b"]} (which becomes allowedFunctionNames); a raw map is passed through. :native_tools adds Vertex's built-in tools alongside your function declarations.

defmodule WeatherTools do
  @doc "Get the current weather for a city"
  def get_weather(%{"city" => city}, _deps) do
    {:ok, "18C and clear in #{city}"}
  end
end

agent =
  Nous.new("vertex_ai:gemini-2.5-flash",
    tools: [&WeatherTools.get_weather/2],
    model_settings: %{
      tool_choice: {:any, ["get_weather"]},
      native_tools: [:google_search]
    }
  )

{:ok, result} = Nous.run(agent, "What is the weather in Kyoto right now?")
result.output

:native_tools entries may be bare atoms (:google_search → %{"googleSearch" => %{}}, :url_context → %{"urlContext" => %{}}, :code_execution → %{"codeExecution" => %{}}), {name, config_map} tuples for tools that take configuration (the name is camelized), or a raw map when you want full control.

When you need a toolConfig shape Nous does not model, set :tool_config directly. It is used verbatim and takes precedence over :tool_choice:

agent =
  Nous.new("gemini:gemini-2.5-flash",
    tools: [&WeatherTools.get_weather/2],
    model_settings: %{
      tool_config: %{
        "functionCallingConfig" => %{
          "mode" => "ANY",
          "allowedFunctionNames" => ["get_weather"]
        }
      }
    }
  )

{:ok, result} = Nous.run(agent, "Weather in Osaka?")
result.output

Sampling and generation config

:top_k, :seed, :candidate_count, and :response_modalities are forwarded into generationConfig unchanged (as topK, seed, candidateCount, responseModalities), next to the cross-provider sampling keys:

agent =
  Nous.new("gemini:gemini-2.5-flash",
    model_settings: %{
      temperature: 0.2,
      max_tokens: 2048,
      top_p: 0.95,
      top_k: 40,
      seed: 42,
      candidate_count: 1,
      response_modalities: ["TEXT"],
      stop_sequences: ["\n\n"]
    }
  )

{:ok, result} = Nous.run(agent, "Give me three product names.")
result.output

Note on :candidate_count: Nous parses only the first candidate of a response (streaming and non-streaming alike), so asking for more than one candidate costs output tokens without surfacing the extra completions.

Context caching

:cached_content is a pass-through of Vertex's top-level cachedContent field. Nous neither creates nor validates caches — create one with the Vertex REST API and reference the resource name it returns:

agent =
  Nous.new("vertex_ai:gemini-2.5-flash",
    model_settings: %{
      cached_content:
        "projects/my-project/locations/us-central1/cachedContents/1234567890"
    }
  )

{:ok, result} = Nous.run(agent, "Given the cached contract, when does it expire?")
result.usage.cache_read_input_tokens

Cache hits come back from Vertex as usageMetadata.cachedContentTokenCount, which Nous parses into usage.cache_read_input_tokens.

Input Validation

The provider validates GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION at request time and returns helpful error messages for invalid values instead of opaque DNS or HTTP errors.

Examples

• examples/providers/vertex_ai.exs — Basic usage with access token
• examples/providers/vertex_ai_goth_test.exs — Service account with Goth
• examples/providers/vertex_ai_multi_region.exs — Multi-region + v1/v1beta1 demo
• examples/providers/vertex_ai_integration_test.exs — Full integration test (Flash + Pro, streaming + non-streaming)