How to Clone Any Voice in Spanish Locally on Linux (CachyOS & Linux Mint) with F5-TTS and a RAM-Resident Daemon
Cloning a voice with AI 100% locally on Linux usually hits two major roadblocks: the default F5-TTS base model speaks Spanish with a heavy English accent unless a proper Spanish checkpoint is configured, and loading a 5+ GB model from disk every single time you want to synthesize a sentence makes the workflow painfully slow.
In this step-by-step guide, you will set up a local Spanish voice cloning TTS engine on CachyOS (Arch Linux) and Linux Mint (Ubuntu/Debian) that solves both problems from the ground up:
Native Spanish pronunciation: Automatically downloads and adapts the
F5TTS_BaseSpanish architecture.Model optimization from 5.4 GB down to 1.3 GB (
.safetensors): Extracts only the inference tensors and discards training overhead (optimizer_state_dict).RAM-Resident UNIX Socket Daemon: Loads the model into RAM once and responds immediately from your terminal (Fish or Bash), playing audio directly through your speakers (
PipeWire/PulseAudio) and letting you export the generated voice to any.wavor.mp3path.
1. Install System Dependencies
We need ffmpeg (to clean up the reference audio and export formats), python, and your system's audio playback utilities.
On CachyOS (Primary)
CachyOS comes with PipeWire (pw-play) and the fish shell out of the box:
sudo pacman -S --needed python python-pip ffmpeg pipewire-audio
On Linux Mint
On Linux Mint, install the equivalent packages via apt:
sudo apt update && sudo apt install -y python3 python3-pip python3-venv ffmpeg pulseaudio-utils
2. Install F5-TTS and Python Libraries
Install f5-tts, safetensors, and faster-whisper (used to automatically transcribe the voice sample you want to clone).
On CachyOS:
Fragmento de códigopip install --user --break-system-packages f5-tts safetensors faster-whisper huggingface_hubOn Linux Mint:
Bashpip3 install --user --break-system-packages f5-tts safetensors faster-whisper huggingface_hub(Note: If you are on an older version of Linux Mint where
--break-system-packagesis not required, you can omit that flag).
3. Prepare Your Voice Sample (6 to 10 Seconds)
For F5-TTS to clone vocal timbre, cadence, and intonation cleanly, two rules are critical:
Ideal duration: A 6-to-10-second clip of clear speech with no background music.
Exact audio-text alignment (
ref_text): The model must know the exact words spoken in the reference audio. If the reference transcript does not match the audio syllable by syllable, the generated output will sound garbled.
Place your original recording on your Desktop named voz.wav (either ~/Escritorio/voz.wav or ~/Desktop/voz.wav).
4. Master Setup Script: Download, .safetensors Conversion & Auto-Transcription
This script handles all the heavy lifting once:
Automatically detects your Desktop folder (
EscritorioorDesktop).Cleans background noise from
voz.wavusingffmpegFFT filters without cutting out your natural breathing pauses, generating a 24,000 Hz monoref_muestra_voz.wav.Transcribes what you say in that clip using Faster-Whisper and saves it to
ref_texto_voz.txt.Downloads the Spanish checkpoint
andromeda0302/E2-F5-TTS-SPANISH, aligns the vocabulary file to preventsize mismatcherrors, and saves a fastf5_es_fast.safetensors(1.3 GB) model.
Run this block in your terminal (works identically in Fish on CachyOS and Bash on Linux Mint):
python3 -c '
import os, subprocess, inspect, httpx, torch
os.environ["PYTHONHASHSEED"] = "42"
os.environ["NETRC"] = "/dev/null"
os.environ["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "1"
_init = httpx.Client.__init__
httpx.Client.__init__ = lambda s, *a, **k: _init(s, *a, **{**k, "trust_env": False})
_send = httpx.Client.send
httpx.Client.send = lambda s, r, *a, **k: (r.headers.pop("authorization", None), _send(s, r, *a, **k))[1]
from pathlib import Path
from safetensors.torch import save_file
from huggingface_hub import hf_hub_download, list_repo_files
import f5_tts.infer.utils_infer as ui
esc = Path.home() / ("Escritorio" if (Path.home() / "Escritorio").exists() else "Desktop")
v = esc / "voz.wav"
m = esc / "ref_muestra_voz.wav"
t = esc / "ref_texto_voz.txt"
fv = esc / "f5_es_vocab.txt"
fast_ckpt = esc / "f5_es_fast.safetensors"
if not v.exists():
raise FileNotFoundError(f"Please place your voice file at: {v}")
print("✂ 1/4 Cleaning and normalizing 8s sample with FFmpeg...")
subprocess.run([
"ffmpeg", "-y", "-i", str(v),
"-af", "highpass=f=80,lowpass=f=10000,afftdn=nf=-20,silenceremove=start_periods=1:start_threshold=-42dB,dynaudnorm=f=150:g=15",
"-ar", "24000", "-ac", "1", "-t", "8", str(m)
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
if not fast_ckpt.exists() or not fv.exists():
repo = "andromeda0302/E2-F5-TTS-SPANISH"
print("⬇ 2/4 Downloading Spanish model from Hugging Face:", repo)
files = list_repo_files(repo, token=False)
cn = [f for f in files if f.endswith((".safetensors", ".pt"))][0]
pt_ckpt = hf_hub_download(repo_id=repo, filename=cn, token=False)
print("⚡ 3/4 Extracting fast model (.safetensors, 1.3 GB)...")
raw = torch.load(pt_ckpt, map_location="cpu", weights_only=False)
sd = raw.get("ema_model_state_dict", raw.get("model_state_dict", raw))
tensors = {k: val.contiguous() for k, val in sd.items() if torch.is_tensor(val)}
ks = list(tensors.keys())
ek = [k for k in ks if "text_embed.weight" in k][0]
vs, _ = tensors[ek].shape
def_v = Path(ui.__file__).resolve().parent / "examples" / "vocab.txt"
v_lines = def_v.read_text(encoding="utf-8").splitlines()
v_lines = v_lines[:vs-1] if len(v_lines) >= vs else v_lines + [f"<extra_{i}>" for i in range(vs - 1 - len(v_lines))]
fv.write_text("\n".join(v_lines) + "\n", encoding="utf-8")
save_file(tensors, str(fast_ckpt))
del raw, sd, tensors
print("📝 4/4 Transcribing your voice sample with Faster-Whisper...")
from faster_whisper import WhisperModel
wm = WhisperModel("small", device="cpu", compute_type="int8")
segs, _ = wm.transcribe(str(m), language="es")
txt = " ".join(s.text.strip() for s in segs).strip()
t.write_text(txt, encoding="utf-8")
print(f"\n✅ SETUP COMPLETE.\n📌 Detected transcript saved to {t.name}: \"{txt}\"")
'
Tip: Check the transcript printed in the terminal. If you want to fix any word or punctuation mark, simply edit
ref_texto_voz.txton your Desktop.
5. Create the Fast TTS Daemon with Audio Export (.wav / .mp3)
Next, we create the executable script at ~/.local/bin/tts_clon.py. This script:
Fixes the Python 3.14
PYTHONHASHSEEDcrash on bleeding-edge distributions like CachyOS.Keeps the model loaded in RAM listening on the UNIX socket
/tmp/tts_clon.sock.Speaks immediately and lets you export the generated voice to any
.wavor.mp3path (saving by default to~/Escritorio/tts_salida.wavor~/Desktop/tts_salida.wav).
Run this command to install the script:
mkdir -p ~/.local/bin && cat << 'EOF' > ~/.local/bin/tts_clon.py
#!/usr/bin/env python3
import os, sys, time, json, socket, shutil, random, subprocess, inspect
os.environ["PYTHONHASHSEED"] = "42"
os.environ["NETRC"] = "/dev/null"
os.environ["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "1"
from pathlib import Path
SOCK = "/tmp/tts_clon.sock"
esc = Path.home() / ("Escritorio" if (Path.home() / "Escritorio").exists() else "Desktop")
default_out = esc / "tts_salida.wav"
def play_audio(file_path):
for cmd in (
["pw-play", str(file_path)],
["paplay", str(file_path)],
["ffplay", "-nostdin", "-nodisp", "-autoexit", "-loglevel", "quiet", str(file_path)],
["aplay", "-q", str(file_path)],
):
if shutil.which(cmd[0]):
subprocess.run(cmd)
break
def run_server():
import torch
torch.set_num_threads(os.cpu_count() or 12)
import f5_tts.api as f5_api
import f5_tts.model.utils as f5_utils
def _safe_seed(seed=42):
s = abs(int(seed)) % 2147483647 if seed is not None else 42
os.environ["PYTHONHASHSEED"] = str(s)
random.seed(s)
torch.manual_seed(s)
f5_api.seed_everything = _safe_seed
f5_utils.seed_everything = _safe_seed
ref_wav = esc / "ref_muestra_voz.wav"
ref_txt_file = esc / "ref_texto_voz.txt"
fv = esc / "f5_es_vocab.txt"
fast_ckpt = esc / "f5_es_fast.safetensors"
params = inspect.signature(f5_api.F5TTS.__init__).parameters
kw = {"ckpt_file": str(fast_ckpt), "vocab_file": str(fv)}
kw.update({"model": "F5TTS_Base"} if "model" in params else {"model_type": "F5-TTS"})
f5 = f5_api.F5TTS(**kw)
if os.path.exists(SOCK):
os.remove(SOCK)
srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
srv.bind(SOCK)
srv.listen(5)
while True:
conn, _ = srv.accept()
try:
data = conn.recv(8192).decode("utf-8")
if not data:
continue
req = json.loads(data)
txt = req["text"].strip()
out = req["out"]
if txt and txt[-1] not in ".!?":
txt += "."
ref_txt = ref_txt_file.read_text(encoding="utf-8").strip()
f5.infer(
ref_file=str(ref_wav),
ref_text=ref_txt,
gen_text=txt,
file_wave=out,
nfe_step=12,
cfg_strength=2.0,
sway_sampling_coef=-1.0,
speed=1.0,
seed=42,
)
conn.sendall(b"OK")
except Exception as e:
conn.sendall(f"ERR:{e}".encode("utf-8"))
finally:
conn.close()
if len(sys.argv) > 1 and sys.argv[1] == "--daemon":
run_server()
sys.exit(0)
if len(sys.argv) < 2:
print("Usage:\n tts_julio \"Text to speak\"\n tts_julio \"Text to speak\" /path/to/export.wav\n tts_julio \"Text to speak\" /path/to/export.mp3")
sys.exit(1)
last_arg = sys.argv[-1].lower()
export_requested = len(sys.argv) >= 3 and last_arg.endswith((".wav", ".mp3", ".ogg", ".flac"))
if export_requested:
texto = " ".join(sys.argv[1:-1]).strip()
final_export_path = Path(sys.argv[-1]).expanduser().resolve()
else:
texto = " ".join(sys.argv[1:]).strip()
final_export_path = default_out
final_export_path.parent.mkdir(parents=True, exist_ok=True)
tmp_wav_path = final_export_path if final_export_path.suffix.lower() == ".wav" else esc / "_tmp_tts_render.wav"
def send_req():
c = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
c.connect(SOCK)
c.sendall(json.dumps({"text": texto, "out": str(tmp_wav_path)}).encode("utf-8"))
return c.recv(1024).decode("utf-8")
try:
res = send_req()
except Exception:
print("⚡ Starting resident TTS engine in RAM (only takes a moment on first run)...")
subprocess.Popen([sys.executable, __file__, "--daemon"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
for _ in range(90):
time.sleep(0.5)
if os.path.exists(SOCK):
try:
res = send_req()
break
except Exception:
pass
else:
raise RuntimeError("Timed out waiting for the TTS daemon to start.")
if res == "OK":
if final_export_path.suffix.lower() != ".wav":
subprocess.run(["ffmpeg", "-y", "-i", str(tmp_wav_path), str(final_export_path)], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
tmp_wav_path.unlink(missing_ok=True)
print(f"💾 Audio exported to: {final_export_path}")
print("🔊 Playing...")
play_audio(final_export_path)
else:
print("❌ Error:", res)
EOF
chmod +x ~/.local/bin/tts_clon.py
6. Register the Command in Your Shell (CachyOS & Linux Mint)
On CachyOS (fish shell)
Create and save the tts_julio function permanently:
function tts_julio
python3 ~/.local/bin/tts_clon.py $argv
end
funcsave tts_julio
On Linux Mint (bash shell)
Add the alias to your ~/.bashrc and reload it:
echo 'alias tts_julio="python3 ~/.local/bin/tts_clon.py"' >> ~/.bashrc
source ~/.bashrc
7. How to Use and Export Your Cloned Voice
You can now run your cloned voice from any directory on your system:
Speak immediately (saves by default to
tts_salida.wavon your Desktop):Fragmento de códigotts_julio "Hola amigos, esta es una prueba de mi voz clonada corriendo localmente en Linux."Export the generated voice to a custom
.wavfile (for Ardour, Kdenlive, or automation scripts):Fragmento de códigotts_julio "Bienvenidos a un nuevo video del canal." ~/Escritorio/intro_video.wavExport directly to
.mp3or.ogg: If the target path ends in.mp3,.ogg, or.flac, the script automatically converts the output viaffmpeg:Fragmento de códigotts_julio "El respaldo de la base de datos ha finalizado correctamente." ~/Escritorio/alerta_sistema.mp3Free up 5.4 GB of Hugging Face cache (Optional): Once
f5_es_fast.safetensorshas been generated on your Desktop, you can safely remove the original 5.4 GB.ptfile from your Hugging Face cache:Fragmento de códigorm -rf ~/.cache/huggingface/hub/models--andromeda0302--E2-F5-TTS-SPANISH

Comments
Post a Comment