Voice AI System with Retell AI

This guide walks you through creating a voice-based AI system using Retell AI, with a focus on voice selection, tone, prompt design, setting up a knowledge base, managing bookings and FAQs, handling call transfers, designing dynamic prompt trees, and integrating with Google Sheets and n8n for real-time updates. Let’s dive in!

1. Getting Started with Retell AI

Retell AI lets you build conversational voice agents for customer service, bookings, and more. Sign up at retell.ai and grab your API keys from the dashboard to get started.

What You’ll Need

• Basic knowledge of JavaScript/Node.js
• A Retell AI account
• Google Sheets for data storage
• n8n for automation (optional)
• A server or cloud platform to host your application

2. Choosing a Voice and Tone

Retell AI offers a range of voices to match your brand. For this guide, we’ll pick a clear, professional US English voice with a friendly tone.

Steps for Voice Selection

1. Explore Voices: In the Retell AI dashboard, go to the “Voices” section to preview options.
2. Select a Voice: Choose a voice ID (e.g., voice_us_english_female_001) and note it for your agent setup.
3. Adjust Tone: Set tone parameters like pitch, speed, and warmth via the API to keep it professional yet approachable.

Example Voice Configuration

{
  "voice_id": "voice_us_english_female_001",
  "tone": {
    "pitch": 1.0,
    "speed": 0.9,
    "warmth": 0.8
  }
}

3. Setting Up a Knowledge Base for Customer Queries

A knowledge base helps your voice agent answer common questions accurately and quickly.

Steps to Create a Knowledge Base

1. Gather FAQs: Collect common customer questions like “What are your hours?” or “How do I cancel a booking?”
2. Organize Data: Store the FAQs in a JSON file or a database like Firebase for easy access.
3. Connect to Retell AI: Use the Retell AI API to link the knowledge base to your agent for query handling.

Example Knowledge Base (JSON)

{
  "faqs": [
    {
      "question": "What are your business hours?",
      "answer": "We’re open Monday through Friday, 9 AM to 5 PM."
    },
    {
      "question": "How do I cancel a booking?",
      "answer": "You can cancel by calling us or using our online portal."
    }
  ]
}

Implementation

Set up a webhook in Retell AI to process incoming queries and match them to your knowledge base using natural language processing (NLP).

4. Managing Bookings, FAQs, and Call Transfers

Your voice agent can handle bookings, answer FAQs, and transfer calls to human agents when needed.

Bookings

• Setup: Use Retell AI’s API to collect booking details (name, date, time) during the call.
• Store Data: Save the information in a database or Google Sheets (covered in Section 6).
• Confirm Booking: Have the agent confirm with the user, e.g., “Your appointment is set for 3 PM tomorrow.”

Call Transfers

• Define Triggers: Set conditions for transfers, like complex queries or user requests for a human.
• Implement: Use Retell AI’s transfer_call API to redirect calls to a human agent’s number.

Example Transfer Code

if (userQuery.includes("speak to a person")) {
  retellClient.transferCall({
    call_id: callId,
    to_number: "+1234567890"
  });
}

5. Designing Dynamic Prompt Trees for Natural Conversations

Dynamic prompt trees guide the conversation, making it feel natural and responsive.

Steps to Design Prompt Trees

1. Plan the Flow: Outline the conversation path—greeting, query handling, resolution, or escalation.
2. Use Conditional Logic: Create branches for different scenarios (e.g., booking, FAQs, or transfers).
3. Add Fallbacks: Include responses for unrecognized inputs, like “Sorry, I didn’t catch that. Could you repeat?”
4. Test and Refine: Use Retell AI’s testing tools to simulate conversations and improve prompts.

Example Prompt Tree (Pseudo-code)

const promptTree = {
  greeting: {
    response: "Hello! Thanks for calling. How can I help you today?",
    next: {
      "book appointment": {
        response: "Great! What date and time would you like?",
        collect: ["date", "time", "name"]
      },
      "question": {
        response: "Please ask your question, and I’ll do my best to help.",
        action: "query_knowledge_base"
      },
      "human": {
        action: "transfer_call"
      }
    }
  }
};

6. Connecting to Google Sheets for Real-Time Updates

Google Sheets is a simple way to store and manage booking data or FAQs with real-time updates.

Steps

1. Set Up Google Sheets API:
   • Create a Google Cloud project and enable the Google Sheets API.
   • Generate credentials (OAuth 2.0 or service account) and download the JSON key.
2. Access Sheets: Use a library like googleapis in Node.js to read and write data.
3. Sync with Retell AI: Send booking data to Google Sheets via the API when a booking is made.

Example Node.js Code for Google Sheets

const { google } = require('googleapis');
const sheets = google.sheets('v4');

async function addBooking(auth, bookingData) {
  const request = {
    spreadsheetId: 'YOUR_SPREADSHEET_ID',
    range: 'Sheet1!A:C',
    valueInputOption: 'RAW',
    insertDataOption: 'INSERT_ROWS',
    resource: {
      values: [[bookingData.name, bookingData.date, bookingData.time]]
    },
    auth: auth
  };
  await sheets.spreadsheets.values.append(request);
}

7. Automating with n8n for Real-Time Workflows

n8n is a powerful tool for automating tasks like updating Google Sheets or sending confirmation emails.

Steps

1. Install n8n: Set it up locally or use the cloud version at n8n.io.
2. Create a Workflow:
   • Trigger: Use a webhook to receive booking data from Retell AI.
   • Actions: Add nodes to update Google Sheets, send emails, or notify your team (e.g., via Slack).
3. Connect to Retell AI: Use n8n’s HTTP Request node to pull call data or push updates.

Example n8n Workflow

• Webhook Node: Listens for booking data from Retell AI.
• Google Sheets Node: Appends the data to your sheet.
• Email Node: Sends a confirmation email to the customer.

Example Webhook Setup

{
  "endpoint": "https://your-n8n-instance/webhook/retell",
  "method": "POST",
  "data": {
    "call_id": "{{callId}}",
    "booking": "{{bookingData}}"
  }
}

8. Putting It All Together

Here’s how the system works:

• The Retell AI agent greets callers with a friendly, professional tone.
• It uses the prompt tree to handle bookings, answer FAQs, or transfer calls.
• Booking details are saved to Google Sheets, with n8n automating additional tasks like notifications.
• If a caller needs a human, the agent transfers the call seamlessly.

9. Tips for Success

• Test Extensively: Run test calls to ensure the agent sounds natural and handles queries correctly.
• Keep It Professional: Use a clear, friendly tone to build trust with users.
• Monitor Performance: Check Retell AI’s call logs to identify and fix issues.
• Secure Data: Use HTTPS for webhooks and encrypt sensitive data in Google Sheets.

Wrapping Up

With Retell AI, Google Sheets, and n8n, you can create a robust voice AI system that handles customer interactions, bookings, and automation with ease. For more details, visit retell.ai. Now you’re ready to build a system that’s as reliable as it is user-friendly!