In today’s rapidly evolving digital landscape, the ability to interact with AI agents through natural language is becoming increasingly vital. While text-based interactions are common, adding a voice interface can significantly enhance user experience, making AI feel more intuitive and accessible. From my experience building automated systems, connecting voice capabilities to your AI agents isn’t as complex as it might seem, especially when leveraging powerful tools like n8n and sophisticated voice AI platforms like ElevenLabs.

Share by Nate Herk

I will guide you through two primary methods for creating voice-enabled AI agents that stand out for their efficiency and versatility:

  1. Asynchronous Voice Interaction via Telegram: This method involves a “send-and-receive” model where a user sends a voice message on Telegram, an AI processes it, and a voice response is sent back.
  2. Real-Time Conversational AI Tool: This method creates a backend “tool” that a live conversational agent can use to perform complex tasks, like real-time research and summarization, triggered by a webhook.
Download workflow: https://romhub.io/n8n/Voice_Workflows

Let’s delve into how you can set up these powerful voice-enabled AI agents, step-by-step, directly following the logic of the workflow.

Method One: Building an Asynchronous Voice-to-Voice Workflow in Telegram

This approach is perfect for scenarios where users send a voice message, your AI processes it, and then sends back an audio reply. The attached workflow automates this entire process seamlessly.

1. Trigger the Workflow and Get the Voice File

The first step is to capture the user’s voice input from Telegram.

  • Telegram Trigger Node: The workflow begins with this node, configured to listen for new messages. When a user sends a voice message to your bot, this node activates the entire sequence.
  • Get a file Node: The trigger only provides metadata. This second Telegram node takes the file_id from the trigger’s output (={{ $json.message.voice.file_id }}) to download the actual audio file into the n8n environment.

2. Transcribe Audio with ElevenLabs

Now that we have the voice file, we must convert speech into text for our AI agent.

  • Transcribe audio or video Node: The workflow uses an ElevenLabs node set to the “speechToText” operation. It takes the binary data from the previous step as input. You will need to connect your ElevenLabs API key to this node for it to function. Upon execution, the output will be the transcribed text.

3. Process the Text with an AI Agent

With the audio now converted to text, it’s time for the AI agent to generate a response.

  • AI Agent Node: The workflow uses a LangChain agent node. Its text input is set to the transcribed text from the ElevenLabs node (={{ $json.text }}).
  • System Prompt: To define the AI’s personality, a specific system prompt is used: “You are a helpful assistant who is extremely funny.”. This guides the AI to generate witty and helpful responses.
  • OpenRouter Chat Model: The AI Agent is powered by an OpenRouter chat model, which requires a separate API key for its credentials.

4. Convert AI Response Back to Speech

The AI agent has responded in text. Now, we turn that text back into an audio file.

  • Convert text to speech Node: Another ElevenLabs node is used, this time for its text-to-speech capabilities.
  • Voice Selection: The workflow specifies a particular voice using its ID (9PVP7ENhDskL0KYHAKtD). You can customize this by selecting a voice from the list or providing a different Voice ID from your ElevenLabs library.
  • Input Text: The node’s text field is mapped to the output from the AI Agent (={{ $json.output }}) to convert the AI’s generated response into audio.

5. Send the Audio Reply via Telegram

The final step is to deliver the AI’s voice response back to the user.

  • Send an audio file Node: This Telegram node is configured to send audio.
  • Chat ID and Data: It dynamically uses the chat_id from the initial Telegram Trigger to ensure the reply is sent to the correct user (={{ $('Telegram Trigger').item.json.message.chat.id }}). It sends the binary audio data generated by the previous ElevenLabs node.

Once all these steps are configured and the n8n workflow is activated, you have a complete, automated voice-to-voice interaction loop within Telegram.

Method Two: Crafting a Real-Time Research Tool for a Conversational AI

This method is more advanced. The n8n workflow acts as a backend tool that a dedicated conversational AI agent (like one from ElevenLabs) can call via a webhook to perform live research and summarization.

1. The Webhook Trigger: The Gateway for Your Tool

The workflow starts when an external service makes a request.

  • Webhook Node: This node is configured to accept POST requests. It acts as the entry point for the ElevenLabs conversational agent. When the agent needs to perform a search, it calls this webhook’s URL. Its response mode is set to use a corresponding “Respond to Webhook” node, meaning it will wait for the entire workflow to finish before sending a reply.

2. Perform Web Research with Perplexity

Once triggered, the workflow immediately begins the research task.

  • Message a model Node (Perplexity): This node connects to Perplexity AI to perform a web search. It is configured to use the “sonar” model.
  • Search Query: The search query is extracted from the body of the incoming webhook request (={{ $json.body.searchQuery }}). This is the information the conversational agent needs to look up.

3. Summarize Results with a Specialized AI Agent

The research results from Perplexity can be lengthy. This step condenses the information into a concise summary suitable for a voice response.

  • AI Agent1 Node: A second, distinct AI agent is used for this task. It takes the message from the Perplexity node as its input (={{ $json.message }}).
  • System Prompt for Summarization: This agent’s behavior is strictly defined by its system prompt: “You are an expert research agent. You will be fed information, and you need to make a concise summary. It should only be about three sentences.”. This ensures the final response is brief and to the point.
  • OpenRouter Chat Model1: This summarization agent is also powered by an OpenRouter model.

4. Respond to the Webhook

The final step in n8n is to send the summarized research back to the calling service.

  • Respond to Webhook Node: This node is placed at the end of the flow. It takes the summarized text generated by AI Agent1 and sends it back as the response to the initial webhook request, completing the tool’s execution loop.

This workflow effectively creates a powerful, on-demand research-and-summarization tool that any conversational AI platform capable of making webhook calls can leverage.

Activating and Securing Your AI Voice Workflows

Once you have configured and tested your workflows based on the provided JSON, it’s essential to prepare them for live use.

  • From Test to Production: For the real-time conversational tool, you must switch from the webhook’s “Test URL” to its “Production URL” within your external application (e.g., the ElevenLabs agent’s tool settings).
  • Activate Your Workflows: For both methods, toggle your n8n workflows to “Active.” This ensures they run continuously in the background and process triggers automatically without manual intervention.
  • Security Best Practices: As highlighted in the workflow’s setup guide, always use n8n’s built-in credential management to securely store your API keys for ElevenLabs, OpenRouter, and Perplexity. Never hardcode sensitive information directly into nodes.

Conclusion: The Future of Interactive AI

By accurately following the two distinct workflows provided, you can integrate sophisticated voice capabilities into your AI agents using n8n and ElevenLabs. Whether you’re building an asynchronous voice chatbot for Telegram or a powerful, real-time research tool for a conversational AI, these platforms provide the necessary flexibility and power.

Understanding how to connect voice input to AI processing, convert text responses back into natural-sounding speech, and enable tool-calling for dynamic research puts you at the forefront of creating truly intuitive AI experiences. The ability to speak to an AI, have it understand, process, research, and respond audibly is a significant leap forward. I encourage you to experiment with these methods, customize the prompts and voices, and discover how voice can transform your AI applications.

You may also like

Subscribe
Notify of
guest

0 Comments
Newest
Oldest Most Voted
Inline Feedbacks
View all comments