## Summary - Migrate from deprecated `langchain-google-vertexai` (gRPC) to `langchain-google-genai` (REST) to fix silent hang issues during model initialization - Add initialization and warm-up timeouts using ThreadPoolExecutor for cross-platform support - Upgrade key logs from DEBUG to INFO with timing information for better visibility ## Problem Job `b1b2648c` hung indefinitely at "LangFuse prompt service initialized" with no visible errors. Root cause: `ChatVertexAI` uses gRPC which can hang during connection establishment with no timeout. ## Changes | File | Changes | |------|---------| | `pyproject.toml` | Replace `langchain-google-vertexai` with `langchain-google-genai` | | `lyrics_transcriber_temp/pyproject.toml` | Same dependency change | | `.../providers/config.py` | Add `initialization_timeout_seconds` (30s) and `warmup_timeout_seconds` (15s) | | `.../providers/model_factory.py` | Migrate to `ChatGoogleGenerativeAI`, add timing logs | | `.../providers/langchain_bridge.py` | Add ThreadPoolExecutor timeout wrapper, upgrade logs to INFO | | `.../agent.py` | Add timing logs for `classify_gap()` | ## Test plan - [x] All 812 tests pass - [x] E2E emulator tests pass - [ ] Test with actual GOOGLE_API_KEY in staging 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Configurable initialization and warmup timeouts. * Support for Google AI Studio (new Google generative API) as a model backend. * **Improvements** * Enhanced logging and timing visibility across model initialization, warm‑up, invocation, and gap classification. * Warm‑up timeouts treated as non‑fatal; initialization timeouts surfaced clearly and reported. * **Bug Fixes** * Better handling/reporting for initialization timeouts and missing Google API key. * **Chores** * Dependency key updated to new Google generative client; package versions bumped. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub> <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
3.2 KiB
LangChain Vertex AI to Google GenAI Migration
Date: 2025-12-30 PR: #145
Problem
Cloud job b1b2648c hung silently during agentic correction. No logs appeared after "LangFuse prompt service initialized". Investigation revealed ChatVertexAI (langchain-google-vertexai) was hanging during gRPC connection establishment with no timeout.
Solution
Migrated from langchain-google-vertexai to langchain-google-genai:
| Before | After |
|---|---|
ChatVertexAI (gRPC-based) |
ChatGoogleGenerativeAI (REST-based) |
| Vertex AI service account auth | GOOGLE_API_KEY (AI Studio) |
| No initialization timeout | 30s init + 15s warm-up timeouts |
| DEBUG-level logs | INFO-level logs with timing |
Key Changes
Dependencies
# Before
langchain-google-vertexai = ">=3.1.1"
# After
langchain-google-genai = ">=2.0.0"
Model Creation
# Before (gRPC, can hang)
from langchain_google_vertexai import ChatVertexAI
model = ChatVertexAI(model=model_name, project=project, location=location)
# After (REST, explicit API key)
from langchain_google_genai import ChatGoogleGenerativeAI
model = ChatGoogleGenerativeAI(
model=model_name,
google_api_key=api_key,
convert_system_message_to_human=True, # Gemini doesn't support system messages
)
Timeout Implementation
Used ThreadPoolExecutor for cross-platform timeout (works on Windows, Linux, macOS):
from concurrent.futures import ThreadPoolExecutor, TimeoutError as FuturesTimeoutError
with ThreadPoolExecutor(max_workers=1) as executor:
future = executor.submit(self._factory.create_chat_model, model, config)
try:
self._chat_model = future.result(timeout=30)
except FuturesTimeoutError:
raise InitializationTimeoutError("Model initialization timed out")
Warm-up Pattern
Added lightweight "warm-up" request after model creation to establish connection before real work:
warm_up_prompt = 'Respond with exactly: {"status":"ready"}'
response = model.invoke([HumanMessage(content=warm_up_prompt)])
Files Modified
| File | Changes |
|---|---|
pyproject.toml |
Replaced dependency, bumped version |
lyrics_transcriber_temp/pyproject.toml |
Same |
.../providers/config.py |
Added initialization_timeout_seconds, warmup_timeout_seconds |
.../providers/model_factory.py |
Migrated to ChatGoogleGenerativeAI, added timing logs |
.../providers/langchain_bridge.py |
Added ThreadPoolExecutor timeout wrapper, warm-up, INFO logs |
.../agent.py |
Added timing logs for classify_gap |
Why REST over gRPC
- Simpler debugging - HTTP requests are easier to trace than gRPC streams
- No connection pooling issues - REST is stateless
- Better timeout behavior - HTTP timeouts are well-understood
- Cloud-friendly - Works better through proxies and load balancers
Authentication
The new approach uses Google AI Studio API keys instead of Vertex AI service accounts:
- Get key: https://aistudio.google.com/app/apikey
- Set env var:
GOOGLE_API_KEY=<your-key> - Benefits: Simpler setup, no IAM configuration needed
Note: This uses Google AI Studio quotas, not GCP billing. For high-volume production use, consider keeping Vertex AI with proper timeout handling.