Voice AI Automation Expert
This skill provides comprehensive expertise in three interconnected platforms for building professional voice AI automation systems: Retell AI (conversational voice agents), Eleven Labs (voice cloning and synthesis), and Go High Level (CRM and automation).
Environment Variables
Before deploying voice AI systems, ensure these environment variables are configured in your .env file:
# Retell AI (Voice Agents)
RETELL_API_KEY=your_retell_api_key
# Eleven Labs (Voice Cloning & TTS)
ELEVEN_LABS_API_KEY=your_eleven_labs_api_key
# Go High Level (CRM)
GHL_API_KEY=your_ghl_api_key
GHL_LOCATION_ID=your_location_id
See .env.template for the full list of required environment variables.
Platform Overview
Retell AI
- Conversational voice agents for phone calls
- Custom functions and webhooks for real-time actions
- Knowledge base (RAG) for contextual responses
- Dynamic variables for personalization
- Batch calling for outbound campaigns
- Integration with GHL and other CRMs
Eleven Labs
- Voice cloning (Instant and Professional)
- Text-to-speech with multiple models (Flash, Turbo, Multilingual)
- Voice settings optimization
- API integration for automation
- Multi-language support
Go High Level
- CRM and contact management
- Workflow automation (triggers, filters, actions)
- Calendar and appointment booking
- SMS, email, and phone communication
- Pipeline and opportunity tracking
- API for custom integrations
GHL Webhook Integration
Webhook Endpoint Configuration
Configure Retell AI to send call events to GHL using webhooks:
Webhook URL Pattern:
https://services.leadconnectorhq.com/hooks/{location_id}/retell-webhook
OR (for custom workflows):
https://hook.us1.make.com/{your_make_webhook_id}
GHL Webhook Payload Structure
{
"event": "call_ended",
"call_id": "call_abc123",
"agent_id": "agent_xyz",
"from_number": "+15551234567",
"to_number": "+15559876543",
"duration_seconds": 180,
"call_status": "completed",
"transcript": "Full call transcript...",
"call_summary": "AI-generated summary...",
"sentiment": "positive",
"custom_data": {
"lead_qualified": true,
"appointment_booked": true,
"appointment_date": "2025-01-15",
"appointment_time": "10:00 AM"
}
}
GHL Workflow Trigger Setup
- Create Inbound Webhook Trigger in GHL Workflows
- Map Fields from Retell payload to GHL contact fields:
from_number→ Contact Phonecall_summary→ Contact Notescustom_data.lead_qualified→ Custom Field
- Add Actions: Update contact, move pipeline, send SMS confirmation
Fallback Handling
When AI Can't Understand
Implement robust fallback handling to ensure callers are never stuck:
Prompt-Level Fallbacks
Add these to your Retell AI agent prompt:
FALLBACK RULES:
- If you don't understand after 2 attempts, say: "I apologize, I'm having trouble understanding. Let me transfer you to a team member who can help."
- If caller is frustrated, say: "I understand this is frustrating. Let me get a human on the line right away."
- If technical terms are unclear, ask: "Could you explain that in a different way?"
- Never say "I don't know" - instead say "Let me find that information for you" and trigger the appropriate function.
ESCALATION TRIGGERS:
- Caller asks for a human/manager 2+ times
- Caller expresses frustration or anger
- Call duration exceeds 5 minutes without resolution
- AI confidence drops below threshold
Function-Based Escalation
{
"name": "transfer_to_human",
"description": "Transfer the call to a human agent when AI cannot resolve the issue",
"url": "https://services.leadconnectorhq.com/hooks/{location_id}/transfer",
"method": "POST",
"parameters": {
"type": "object",
"properties": {
"reason": {"type": "string", "description": "Why the transfer is needed"},
"caller_phone": {"type": "string"},
"call_summary": {"type": "string"},
"urgency": {"type": "string", "enum": ["low", "medium", "high"]}
}
}
}
Fallback Response Templates
| Scenario | Response |
|---|---|
| Didn't understand | "I didn't quite catch that. Could you repeat or rephrase?" |
| Still unclear after 2x | "I apologize for the confusion. Let me connect you with someone who can help." |
| Technical question | "That's a great question. Let me check on that for you." (trigger knowledge base) |
| Out of scope | "That's outside what I can help with today. Would you like me to have someone call you back?" |
| Caller frustrated | "I completely understand your frustration. Let me get this resolved for you right away." |
Eleven Labs Voice Cloning Workflow
Step 1: Prepare Voice Samples
Requirements for Quality Cloning:
- 1-5 minutes of clean audio (3+ minutes recommended)
- Single speaker, no background noise
- Natural conversational tone
- WAV or MP3 format, 44.1kHz minimum
- Varied intonation (not monotone)
Recording Best Practices:
DO:
- Use a quiet room with soft furnishings
- Speak at normal pace and volume
- Include varied sentences (questions, statements, exclamations)
- Record multiple takes and use the best
DON'T:
- Record in echoey rooms
- Include "um", "uh", or long pauses
- Use processed or filtered audio
- Rush through the recording
Step 2: Create Voice Clone in Eleven Labs
- Navigate to: Voice Lab → Add Voice → Instant Voice Cloning
- Upload samples: Drag and drop your audio files
- Name the voice: Use format
[Client]-[Purpose]-v1(e.g.,WhiteGlove-Receptionist-v1) - Add description: Include use case and any notes about the voice
- Generate: Click "Add Voice" and wait 30-60 seconds
Step 3: Optimize Voice Settings
| Setting | Recommended | Purpose |
|---|---|---|
| Stability | 0.5 - 0.7 | Higher = more consistent, lower = more expressive |
| Similarity | 0.75 - 0.85 | Higher = closer to original, but can be less natural |
| Style | 0.0 - 0.3 | Add subtle expressiveness (use sparingly) |
| Speaker Boost | On | Enhances clarity for phone calls |
Step 4: Test and Iterate
# Test script for voice quality
import requests
ELEVEN_LABS_API_KEY = "your_api_key"
VOICE_ID = "your_voice_id"
test_phrases = [
"Hello, thank you for calling. How can I help you today?",
"I'd be happy to schedule that appointment for you.",
"Is there anything else I can assist you with?",
"Thank you for choosing us. Have a great day!"
]
for phrase in test_phrases:
response = requests.post(
f"https://api.elevenlabs.io/v1/text-to-speech/{VOICE_ID}",
headers={"xi-api-key": ELEVEN_LABS_API_KEY},
json={
"text": phrase,
"model_id": "eleven_turbo_v2",
"voice_settings": {
"stability": 0.6,
"similarity_boost": 0.8
}
}
)
# Save and review each audio file
Step 5: Connect to Retell AI
- In Retell Dashboard → Agents → Select Agent
- Voice Settings → Custom Voice
- Select "Eleven Labs" as provider
- Enter Voice ID from Eleven Labs
- Test with a sample call
Common Integration Patterns
Inbound Call Handling
Incoming Call → Retell AI Agent → Qualification
↓
Custom Function → GHL API
↓
Create/Update Contact → Add to Pipeline
↓
Book Appointment or Send Follow-up
Outbound Calling Campaign
GHL Contact List → Batch Call API → Retell AI
↓
Dynamic Variables (name, context)
↓
Call Outcome → Webhook to GHL
↓
Update Contact → Move in Pipeline
Testing Checklist
Pre-Deployment Testing
Retell AI Agent
- Test greeting with different caller scenarios
- Verify all custom functions are triggering correctly
- Test fallback responses (mumble, silence, off-topic)
- Confirm knowledge base returns accurate answers
- Test edge cases (interruptions, name spelling, accents)
- Verify call transfer/escalation works
- Check latency is acceptable (under 1 second response)
Eleven Labs Voice
- Listen to test phrases for naturalness
- Check pronunciation of business-specific terms
- Verify voice consistency across different phrases
- Test voice quality over phone (not just speakers)
- Compare against original samples for similarity
GHL Integration
- Verify webhook receives all call events
- Confirm contact creation/update works
- Test pipeline stage movements
- Verify appointment booking syncs to calendar
- Check SMS/email follow-ups trigger correctly
- Test with real phone numbers (not just test mode)
Post-Deployment Monitoring
- Set up daily call log review (first 2 weeks)
- Monitor fallback/transfer rates (target: under 10%)
- Track average call duration
- Review caller satisfaction (if surveyed)
- Monitor API costs across platforms
- Check for failed webhook deliveries
Best Practices
Prompt Engineering for Voice AI
- Keep prompts conversational and natural
- Define clear objectives and boundaries
- Include fallback responses for edge cases
- Test with various caller scenarios
- Optimize for latency (shorter responses)
Voice Quality Optimization
- Record high-quality samples for cloning
- Use appropriate model for use case (Flash for speed, Multilingual for quality)
- Adjust stability and similarity settings
- Test voice in realistic scenarios
Integration Best Practices
- Use webhooks for real-time updates
- Implement error handling and retries
- Log all interactions for debugging
- Monitor costs across platforms
- Test end-to-end flows thoroughly
Cost Optimization
Retell AI
- Use shorter responses to reduce audio minutes
- Batch calls during off-peak hours
- Optimize custom function latency
- Use knowledge base for common questions
Eleven Labs
- Choose Flash model for speed-critical applications
- Use Turbo for balance of quality and cost
- Cache frequently used audio clips
- Monitor character usage
Go High Level
- Use filters to reduce unnecessary workflow triggers
- Batch operations when possible
- Archive inactive contacts
- Monitor API usage limits
Troubleshooting Guide
Call Quality Issues
- Check voice settings (stability, similarity)
- Verify audio format and bitrate
- Test with different voice models
- Check network latency
Integration Failures
- Verify API credentials (RETELL_API_KEY, ELEVEN_LABS_API_KEY, GHL_API_KEY)
- Check webhook URLs and payloads
- Review error logs
- Test with simple requests first
GHL Workflow Issues
- Verify trigger conditions
- Check filter logic
- Review action configurations
- Test with manual triggers
Fallback Issues
- Review call transcripts for patterns
- Adjust prompt fallback thresholds
- Add more specific fallback phrases
- Consider adding more custom functions