Build a self-hosted AI clone with the Hermes agent and Venice

Why Hermes and its persistent memory matter for a clone

Hermes is an open-source agent that has drawn a lot of attention recently, crossing 100,000 GitHub stars in about 30 days. The feature that makes this project work is persistent memory. Hermes does not just hold a single session, it builds a picture of you over time, adapting how it talks and what skills it uses based on what you ask for.

That memory is the hook for the whole experiment. Instead of only learning your patterns to talk back to you, the goal here is to make the agent act as a stand-in for you: matching how you write, how you sound, and how you look. The rest of the guide is the concrete setup to get there.

Venice API: many models behind one endpoint

The Venice API exposes over 300 models across text, image, video, audio, voice cloning, and embeddings, and it can also call tools like web search, web scraping, crypto RPCs, and file uploads. Because everything sits behind one endpoint and one key, the agent can reach any of those capabilities without juggling separate provider accounts.

That breadth is what makes a single agent able to clone a voice, analyze video frames, and generate new video. It can also use Grok models through the Venice API to search X/Twitter, which is how you could later have the agent study your own posting style.

Choosing where to host and provisioning a server

You can run Hermes locally on your laptop or in a Docker container. The tradeoff is availability: if you want the agent reachable on Telegram around the clock, it needs to live somewhere that stays online. For that reason this build uses a VPS.

The demo uses a hosting plan with a Hermes-specific template already in place, so the machine arrives mostly ready. The example plan has 2 vCPU cores, 8 GB of RAM, 100 GB of disk, and 8 TB of bandwidth, and runs about $14 per month. When deploying you can set the billing period to a single month to try it out. Once the machine is up, open root access over SSH to start working on it.

Installing Hermes and connecting the Venice API

With SSH access to the server, install Hermes using the curl command shown in the Venice docs. After it installs, run the quick setup and choose to configure a custom endpoint.

For the custom endpoint, use the Venice API base URL and paste your API key:

https://api.venice.ai/api/v1

To generate a key, go to Venice.ai, open the API page, and create one there. Once the key is in, Hermes lists the available text models. The video selects Kimi K2.6, a newer model that runs as a private, zero-data-retention model on the Venice API, meaning your prompts are not stored. You can leave the model field blank to auto-detect, and set a display name (the demo uses "Venice").

Connecting Telegram and bringing the agent online

To reach the agent from anywhere, enable messaging during setup and connect Telegram. You will need two things: a bot token and your own Telegram ID. Create the bot token in Telegram by talking to BotFather, then paste the token and your Telegram ID into the Hermes prompts.

Install the gateway service so the agent keeps running and stays reachable on Telegram. After setup completes, the agent is live. In the demo it is named Nova and is running on Kimi K2.6 through the Venice API, responding to messages directly in Telegram.

Loading Venice skills and building the clone

To let the agent use the full Venice API, point it at the Venice skills repository on GitHub (Venice AI / skills). Nova pulls in all 19 skills, covering chat, video, music, speech, and transcription, among others, and stores them in memory.

From there, the clone is built by feeding the agent source material and iterating. The demo hands Nova two existing video files and asks it to extract the audio to clone the voice using the Venice voice endpoint, use a vision model to pick the best frames, and run an image-to-video model with lip-sync driven by the cloned voice. The final result used WAN 2.7 for image-to-video, Chatterbox HD for voice cloning, and Grok 4.2 for frame selection, with the voice trained on about two minutes of audio.

Expect manual back-and-forth at the start: adjusting which frame is used, how the voice sounds, fixing odd mouth movement or head motion. The payoff of persistent memory is that once you tune it to your taste, the agent remembers those preferences and applies them to future videos. The demo also switches to a stronger model (Opus 4.7) for the heavier reasoning while the agent still knows little about you, to get more right on the first try.

Key takeaways

• Hermes' persistent memory is what lets an agent stop being a chatbot and start acting as a consistent stand-in for you.
• One Venice API key and base URL gives a single agent access to 300+ models plus tools like web search and voice cloning.
• A VPS keeps the agent online 24/7 on Telegram; local or Docker hosting works if you do not need constant availability.
• Loading the Venice skills repo turns the agent into a full pipeline for voice cloning, frame selection, and lip-synced video, though tuning still takes manual effort.