​

Voice & Speech Configuration

​

Overview

1. Text-to-Speech (TTS): Converts your agent’s responses into natural-sounding audio
2. Speech-to-Text (STT/ASR): Transcribes user speech into text for the LLM to process

​

What You’ll Configure

• TTS Provider: Choose from ElevenLabs, Cartesia, Dasha, Inworld, or LMNT
• Voice Selection: Pick from hundreds of voices across languages and accents
• Voice Model: Select the synthesis model (quality vs. speed tradeoff)
• Speech Speed: Adjust playback rate (provider-dependent)
• Provider-Specific Options: Fine-tune voice characteristics

​

ASR Configuration

• User’s detected language and accent
• Network conditions and latency
• Call quality metrics
• Provider availability

​

TTS Provider Comparison

​

Quick Reference Table

​

Provider Deep Dive

ttsConfig: {
  version: "v1",
  vendor: "ElevenLabs",
  voiceId: "21m00Tcm4TlvDq8ikWAM", // Rachel
  model: "eleven_turbo_v2_5",
  speed: 1.0,
  vendorSpecificOptions: {
    similarity_boost: 0.8,
    stability: 0.6,
    style: 0.4,
    use_speaker_boost: true,
    optimize_streaming_latency: 3
  }
}

vendorSpecificOptions: {
  emotions: [
    "positivity:high",
    "curiosity:low"
  ]
}

ttsConfig: {
  version: "v1",
  vendor: "Cartesia",
  voiceId: "cartesia-voice-id",
  model: "sonic",
  speed: 1.2,
  vendorSpecificOptions: {
    emotions: [
      "positivity:high",
      "curiosity:low"
    ]
  }
}

ttsConfig: {
  version: "v1",
  vendor: "Dasha",
  voiceId: "dasha-voice-id",
  model: "common",
  speed: 1.0
}

ttsConfig: {
  version: "v1",
  vendor: "Inworld",
  voiceId: "inworld-voice-id",
  model: "inworld-tts-1",
  speed: 1.0,
  vendorSpecificOptions: {
    temperature: 0.9,
    pitch: 0.2
  }
}

ttsConfig: {
  version: "v1",
  vendor: "Lmnt",
  voiceId: "lmnt-voice-id",
  model: "blizzard"
  // Note: No speed control available
}

​

Voice Selection

​

Browsing Available Voices

1. Navigate: Go to agent creation/editing → Voice & Speech tab
2. Select Provider: Choose your TTS provider first
3. Browse Voices: Search and filter available voices
4. Preview: Listen to voice samples before selecting
5. Select: Choose your preferred voice

​

Voice Attributes

• Name: Voice identifier (e.g., “Rachel”, “Mark”)
• Language: Primary language and locale (e.g., “en-US”, “es-ES”)
• Gender: Male, Female, or Neutral
• Description: Voice characteristics and use cases
• Provider: TTS service (ElevenLabs, Cartesia, etc.)

​

Custom Voice IDs

1. Find the voice ID from your TTS provider’s documentation
2. Click “Use custom voice ID” button below the voice selector
3. Enter the custom voice ID in the dialog that appears
4. Preview to verify the voice works correctly
5. Save your agent configuration

​

Voice Preview

​

Using Voice Preview

1. Enter Preview Text: Type or paste sample text (up to 1000 characters)
2. Configure Settings: Adjust speed, options as needed
3. Click Preview: Generate and listen to audio sample
4. Iterate: Try different voices, speeds, and settings
5. Select Best Match: Choose the configuration that sounds best

​

Preview Best Practices

• Use representative samples from your agent’s actual responses
• Include questions, statements, and conversational phrases
• Test punctuation handling (commas, periods, exclamation points)
• Try names, numbers, and special terms your agent will use

• Natural pronunciation of domain-specific terms
• Appropriate pacing for your use case
• Emotional tone matches your brand
• Clarity at different speeds
• Consistent quality across phrases

​

Character Limit

• Real-time character counter shows remaining characters
• Warning displays when approaching limit
• Limit enforced by VOICE_PREVIEW_TEXT_LIMIT constant

​

Speed Configuration

​

Speed Ranges by Provider

​

Choosing the Right Speed

• Accessibility needs
• Complex information delivery
• Educational content
• Non-native speakers
• Legal/compliance disclosures

• General conversation
• Customer support
• Sales calls
• Most use cases

• Time-sensitive scenarios
• Familiar/repetitive content
• High-volume information
• Experienced users

​

Voice Cloning

​

Cloning Process

1. Prepare Audio: Record 1-5 minutes of clean voice samples
2. Upload: Use the voice cloning interface or API
3. Configure: Set name, description, language
4. Clone: ElevenLabs processes your samples
5. Use: Select cloned voice in agent configuration

​

API-Based Voice Cloning

// Clone a new voice
const formData = new FormData();
formData.append('Name', 'My Brand Voice');
formData.append('Description', 'Custom voice for customer support');
formData.append('Language', 'en-US');
formData.append('Provider', 'ElevenLabs');
formData.append('audioFiles', audioFile); // File object

const response = await fetch('https://blackbox.dasha.ai/api/v1/voice/clone', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY'
  },
  body: formData
});

const clonedVoice = await response.json();
console.log('Cloned voice ID:', clonedVoice.voiceId);

​

Managing Cloned Voices

// Update cloned voice description
const voiceId = 'your-cloned-voice-id';
await fetch(`https://blackbox.dasha.ai/api/v1/voice/clone/${voiceId}`, {
  method: 'PATCH',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    description: 'Updated voice description'
  })
});

// Delete cloned voice
await fetch(`https://blackbox.dasha.ai/api/v1/voice/clone/${voiceId}`, {
  method: 'DELETE',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY'
  }
});

​

Speech Recognition (ASR)

​

How ASR Works in BlackBox

1. Language Detection: Identifies user’s spoken language
2. Accent Recognition: Adjusts for regional variations
3. Network Quality: Adapts to connection conditions
4. Provider Performance: Routes to best-performing provider
5. Real-Time Optimization: Switches providers if quality degrades

​

Supported ASR Providers

• Deepgram: High-accuracy, low-latency transcription
• Microsoft Speech Services: Enterprise-grade recognition
• Auto (Platform-Managed): Automatic provider selection (recommended)

​

ASR Quality Factors

• Clear microphone input
• Minimal background noise
• Good network connection
• Proper audio levels

• Speech clarity and pace
• Accent and pronunciation
• Use of domain-specific terms
• Speaking patterns

• Automatic noise cancellation
• Echo suppression
• Acoustic model adaptation
• Real-time quality monitoring

​

Improving ASR Accuracy

1. Agent Prompting: Guide users to speak clearly in your agent’s greeting
2. Confirmation: Have agent repeat understood information for verification
3. Clarification: Prompt for clarification when confidence is low
4. Domain Terms: Provide common terms in your agent’s context (coming soon)

​

Configuration Examples

​

Basic Configuration (Recommended)

// Via Dashboard: Use defaults in Voice & Speech tab
// Via API:
const agent = await fetch('https://blackbox.dasha.ai/api/v1/agents', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    name: "Support Agent",
    config: {
      version: "v1",
      primaryLanguage: "en-US",
      ttsConfig: {
        version: "v1",
        vendor: "ElevenLabs",
        voiceId: "zmcVlqmyk3Jpn5AVYcAL",
        model: "eleven_flash_v2_5",
        speed: 1.0
      },
      sttConfig: {
        version: "v1",
        vendor: "Auto" // Platform manages ASR automatically
      },
      llmConfig: {
        version: "v1",
        vendor: "openai",
        model: "gpt-4.1-mini",
        prompt: "You are a helpful assistant."
      },
      features: {
        version: "v1",
        languageSwitching: {
          version: "v1",
          isEnabled: false
        },
        rag: {
          version: "v1",
          isEnabled: false,
          kbLinks: []
        }
      }
    }
  })
});

​

ElevenLabs with Customization

ttsConfig: {
  version: "v1",
  vendor: "ElevenLabs",
  voiceId: "21m00Tcm4TlvDq8ikWAM", // Rachel
  model: "eleven_turbo_v2_5",
  speed: 1.1,
  vendorSpecificOptions: {
    similarity_boost: 0.85,
    stability: 0.7,
    style: 0.4,
    use_speaker_boost: true,
    optimize_streaming_latency: 3
  }
}

​

Cartesia with Emotions

ttsConfig: {
  version: "v1",
  vendor: "Cartesia",
  voiceId: "cartesia-friendly-voice",
  model: "sonic",
  speed: 1.2,
  vendorSpecificOptions: {
    emotions: [
      "positivity:high",
      "curiosity:low"
    ]
  }
}

​

Multilingual Configuration

config: {
  version: "v1",
  primaryLanguage: "en-US", // Default language
  ttsConfig: {
    version: "v1",
    vendor: "ElevenLabs",
    voiceId: "multilingual-voice-id",
    model: "eleven_multilingual_v2", // Supports 29+ languages
    speed: 1.0
  },
  features: {
    version: "v1",
    languageSwitching: {
      version: "v1",
      isEnabled: true
    }
  }
}

​

Testing Voice Configuration

​

Dashboard Testing

1. Save Agent: Save your voice configuration
2. Open Test Widget: Click “Test Agent” in dashboard
3. Start Conversation: Begin voice interaction
4. Listen Carefully: Evaluate voice quality, speed, clarity
5. Iterate: Adjust settings and re-test as needed

​

Voice Synthesis API Testing

// API endpoint: http://localhost:8080 (development) or your production URL
// Synthesize speech using a public ElevenLabs voice
const response = await fetch('https://blackbox.dasha.ai/api/v1/voice/synthesize', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json'
  },
  body: JSON.stringify({
    text: "Hello world, this is a test of the voice synthesis system.",
    provider: "ElevenLabs",
    voiceId: "21m00Tcm4TlvDq8ikWAM", // Rachel (public voice)
    model: "eleven_turbo_v2_5",
    language: "en-US",
    speed: 1.0
  })
});

if (response.ok) {
  const audioBlob = await response.blob();
  const audioUrl = URL.createObjectURL(audioBlob);
  const audio = new Audio(audioUrl);
  audio.play();
}

​

What to Test

• Voice sounds natural and professional
• Speed is comfortable for target audience
• Pronunciation of key terms is correct
• Emotional tone matches use case
• No audio artifacts or glitches
• Consistent quality across phrases
• Latency is acceptable for real-time conversation

​

Best Practices

​

Voice Selection

1. Match Your Brand: Choose voices that align with your brand identity
2. Consider Audience: Select demographics-appropriate voices
3. Test Multiple Options: Preview 3-5 voices before deciding
4. Get Feedback: Test with representative users
5. Document Choice: Note why you selected specific voices

​

Speed Settings

1. Start at 1.0x: Use default speed as baseline
2. Test Incrementally: Adjust in 0.05x increments
3. Context Matters: Different content may need different speeds
4. A/B Test: Compare speeds with real users
5. Monitor Feedback: Track user satisfaction metrics

​

Provider Selection

• You need highest voice quality
• Brand-specific voice cloning is important
• Advanced customization is required
• You can tolerate slightly higher latency

• Real-time responsiveness is critical
• Emotional expression is important
• You need ultra-low latency (< 250ms)
• Conversation feels more important than absolute quality

• You want simple, reliable configuration
• Platform integration is a priority
• You need widest speed adjustment range
• Quick setup is important

• You’re building character-driven experiences
• Gaming or interactive media is your use case
• Voice expressiveness is critical

• You need consistent, predictable output
• Minimal configuration is desired
• Fixed speed (1.0x) works for your needs

​

Common Mistakes to Avoid

​

Troubleshooting

​

Voice Issues

• Solution: Try a different voice from the same provider
• Solution: Adjust stability/temperature parameters (ElevenLabs/Inworld)
• Solution: Switch to ElevenLabs for highest quality

• Solution: Adjust speed setting incrementally
• Solution: Test with representative users
• Solution: Consider accessibility needs (slower may be better)

• Solution: Use phonetic spelling in system prompt
• Solution: Try different voice from same provider
• Solution: Consider voice cloning with correct pronunciation

• Solution: Switch to Cartesia for lowest latency
• Solution: Use ElevenLabs with optimize_streaming_latency: 4
• Solution: Check network conditions

​

ASR Issues

• Solution: No action needed - ASR auto-optimizes
• Solution: Prompt users to speak clearly in agent greeting
• Solution: Add confirmation/verification in conversation flow

• Solution: Ensure agent primaryLanguage matches user language
• Solution: Enable language switching if supporting multiple languages

​

Next Steps

• Test Voice Settings - Detailed voice testing guide
• Voice Cloning - Create custom branded voices
• Advanced Features - Language switching and more
• Tools & Functions - Add capabilities to your agent

​

API Cross-References

• GET /api/v1/voice - List all available voices
• POST /api/v1/voice/synthesize - Synthesize speech for testing
• POST /api/v1/voice/clone - Clone custom voices
• PATCH /api/v1/voice/clone/{voiceId} - Update cloned voice
• DELETE /api/v1/voice/clone/{voiceId} - Delete cloned voice