How to Clone Any Voice in Spanish Locally on Linux (CachyOS & Linux Mint) with F5-TTS and a RAM-Resident Daemon

 

Gemini Generated Image 1vv2131vv2131vv2


Cloning a voice with AI 100% locally on Linux usually hits two major roadblocks: the default F5-TTS base model speaks Spanish with a heavy English accent unless a proper Spanish checkpoint is configured, and loading a 5+ GB model from disk every single time you want to synthesize a sentence makes the workflow painfully slow.

In this step-by-step guide, you will set up a local Spanish voice cloning TTS engine on CachyOS (Arch Linux) and Linux Mint (Ubuntu/Debian) that solves both problems from the ground up:

  1. Native Spanish pronunciation: Automatically downloads and adapts the F5TTS_Base Spanish architecture.

  2. Model optimization from 5.4 GB down to 1.3 GB (.safetensors): Extracts only the inference tensors and discards training overhead (optimizer_state_dict).

  3. RAM-Resident UNIX Socket Daemon: Loads the model into RAM once and responds immediately from your terminal (Fish or Bash), playing audio directly through your speakers (PipeWire / PulseAudio) and letting you export the generated voice to any .wav or .mp3 path.

1. Install System Dependencies

We need ffmpeg (to clean up the reference audio and export formats), python, and your system's audio playback utilities.

On CachyOS (Primary)

CachyOS comes with PipeWire (pw-play) and the fish shell out of the box:

Fragmento de código
sudo pacman -S --needed python python-pip ffmpeg pipewire-audio

On Linux Mint

On Linux Mint, install the equivalent packages via apt:

Bash
sudo apt update && sudo apt install -y python3 python3-pip python3-venv ffmpeg pulseaudio-utils

2. Install F5-TTS and Python Libraries

Install f5-tts, safetensors, and faster-whisper (used to automatically transcribe the voice sample you want to clone).

  • On CachyOS:

    Fragmento de código
    pip install --user --break-system-packages f5-tts safetensors faster-whisper huggingface_hub
    
  • On Linux Mint:

    Bash
    pip3 install --user --break-system-packages f5-tts safetensors faster-whisper huggingface_hub
    

    (Note: If you are on an older version of Linux Mint where --break-system-packages is not required, you can omit that flag).

3. Prepare Your Voice Sample (6 to 10 Seconds)

For F5-TTS to clone vocal timbre, cadence, and intonation cleanly, two rules are critical:

  1. Ideal duration: A 6-to-10-second clip of clear speech with no background music.

  2. Exact audio-text alignment (ref_text): The model must know the exact words spoken in the reference audio. If the reference transcript does not match the audio syllable by syllable, the generated output will sound garbled.

Place your original recording on your Desktop named voz.wav (either ~/Escritorio/voz.wav or ~/Desktop/voz.wav).

4. Master Setup Script: Download, .safetensors Conversion & Auto-Transcription

This script handles all the heavy lifting once:

  • Automatically detects your Desktop folder (Escritorio or Desktop).

  • Cleans background noise from voz.wav using ffmpeg FFT filters without cutting out your natural breathing pauses, generating a 24,000 Hz mono ref_muestra_voz.wav.

  • Transcribes what you say in that clip using Faster-Whisper and saves it to ref_texto_voz.txt.

  • Downloads the Spanish checkpoint andromeda0302/E2-F5-TTS-SPANISH, aligns the vocabulary file to prevent size mismatch errors, and saves a fast f5_es_fast.safetensors (1.3 GB) model.

Run this block in your terminal (works identically in Fish on CachyOS and Bash on Linux Mint):

Bash
python3 -c '
import os, subprocess, inspect, httpx, torch
os.environ["PYTHONHASHSEED"] = "42"
os.environ["NETRC"] = "/dev/null"
os.environ["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "1"

_init = httpx.Client.__init__
httpx.Client.__init__ = lambda s, *a, **k: _init(s, *a, **{**k, "trust_env": False})
_send = httpx.Client.send
httpx.Client.send = lambda s, r, *a, **k: (r.headers.pop("authorization", None), _send(s, r, *a, **k))[1]

from pathlib import Path
from safetensors.torch import save_file
from huggingface_hub import hf_hub_download, list_repo_files
import f5_tts.infer.utils_infer as ui

esc = Path.home() / ("Escritorio" if (Path.home() / "Escritorio").exists() else "Desktop")
v = esc / "voz.wav"
m = esc / "ref_muestra_voz.wav"
t = esc / "ref_texto_voz.txt"
fv = esc / "f5_es_vocab.txt"
fast_ckpt = esc / "f5_es_fast.safetensors"

if not v.exists():
    raise FileNotFoundError(f"Please place your voice file at: {v}")

print("✂ 1/4 Cleaning and normalizing 8s sample with FFmpeg...")
subprocess.run([
    "ffmpeg", "-y", "-i", str(v),
    "-af", "highpass=f=80,lowpass=f=10000,afftdn=nf=-20,silenceremove=start_periods=1:start_threshold=-42dB,dynaudnorm=f=150:g=15",
    "-ar", "24000", "-ac", "1", "-t", "8", str(m)
], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)

if not fast_ckpt.exists() or not fv.exists():
    repo = "andromeda0302/E2-F5-TTS-SPANISH"
    print("⬇ 2/4 Downloading Spanish model from Hugging Face:", repo)
    files = list_repo_files(repo, token=False)
    cn = [f for f in files if f.endswith((".safetensors", ".pt"))][0]
    pt_ckpt = hf_hub_download(repo_id=repo, filename=cn, token=False)
    print("⚡ 3/4 Extracting fast model (.safetensors, 1.3 GB)...")
    raw = torch.load(pt_ckpt, map_location="cpu", weights_only=False)
    sd = raw.get("ema_model_state_dict", raw.get("model_state_dict", raw))
    tensors = {k: val.contiguous() for k, val in sd.items() if torch.is_tensor(val)}
    ks = list(tensors.keys())
    ek = [k for k in ks if "text_embed.weight" in k][0]
    vs, _ = tensors[ek].shape
    def_v = Path(ui.__file__).resolve().parent / "examples" / "vocab.txt"
    v_lines = def_v.read_text(encoding="utf-8").splitlines()
    v_lines = v_lines[:vs-1] if len(v_lines) >= vs else v_lines + [f"<extra_{i}>" for i in range(vs - 1 - len(v_lines))]
    fv.write_text("\n".join(v_lines) + "\n", encoding="utf-8")
    save_file(tensors, str(fast_ckpt))
    del raw, sd, tensors

print("📝 4/4 Transcribing your voice sample with Faster-Whisper...")
from faster_whisper import WhisperModel
wm = WhisperModel("small", device="cpu", compute_type="int8")
segs, _ = wm.transcribe(str(m), language="es")
txt = " ".join(s.text.strip() for s in segs).strip()
t.write_text(txt, encoding="utf-8")
print(f"\n✅ SETUP COMPLETE.\n📌 Detected transcript saved to {t.name}: \"{txt}\"")
'

Tip: Check the transcript printed in the terminal. If you want to fix any word or punctuation mark, simply edit ref_texto_voz.txt on your Desktop.

5. Create the Fast TTS Daemon with Audio Export (.wav / .mp3)

Next, we create the executable script at ~/.local/bin/tts_clon.py. This script:

  • Fixes the Python 3.14 PYTHONHASHSEED crash on bleeding-edge distributions like CachyOS.

  • Keeps the model loaded in RAM listening on the UNIX socket /tmp/tts_clon.sock.

  • Speaks immediately and lets you export the generated voice to any .wav or .mp3 path (saving by default to ~/Escritorio/tts_salida.wav or ~/Desktop/tts_salida.wav).

Run this command to install the script:

Bash
mkdir -p ~/.local/bin && cat << 'EOF' > ~/.local/bin/tts_clon.py
#!/usr/bin/env python3
import os, sys, time, json, socket, shutil, random, subprocess, inspect
os.environ["PYTHONHASHSEED"] = "42"
os.environ["NETRC"] = "/dev/null"
os.environ["HF_HUB_DISABLE_IMPLICIT_TOKEN"] = "1"
from pathlib import Path

SOCK = "/tmp/tts_clon.sock"
esc = Path.home() / ("Escritorio" if (Path.home() / "Escritorio").exists() else "Desktop")
default_out = esc / "tts_salida.wav"

def play_audio(file_path):
    for cmd in (
        ["pw-play", str(file_path)],
        ["paplay", str(file_path)],
        ["ffplay", "-nostdin", "-nodisp", "-autoexit", "-loglevel", "quiet", str(file_path)],
        ["aplay", "-q", str(file_path)],
    ):
        if shutil.which(cmd[0]):
            subprocess.run(cmd)
            break

def run_server():
    import torch
    torch.set_num_threads(os.cpu_count() or 12)
    import f5_tts.api as f5_api
    import f5_tts.model.utils as f5_utils
    def _safe_seed(seed=42):
        s = abs(int(seed)) % 2147483647 if seed is not None else 42
        os.environ["PYTHONHASHSEED"] = str(s)
        random.seed(s)
        torch.manual_seed(s)
    f5_api.seed_everything = _safe_seed
    f5_utils.seed_everything = _safe_seed

    ref_wav = esc / "ref_muestra_voz.wav"
    ref_txt_file = esc / "ref_texto_voz.txt"
    fv = esc / "f5_es_vocab.txt"
    fast_ckpt = esc / "f5_es_fast.safetensors"

    params = inspect.signature(f5_api.F5TTS.__init__).parameters
    kw = {"ckpt_file": str(fast_ckpt), "vocab_file": str(fv)}
    kw.update({"model": "F5TTS_Base"} if "model" in params else {"model_type": "F5-TTS"})
    f5 = f5_api.F5TTS(**kw)

    if os.path.exists(SOCK):
        os.remove(SOCK)
    srv = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    srv.bind(SOCK)
    srv.listen(5)

    while True:
        conn, _ = srv.accept()
        try:
            data = conn.recv(8192).decode("utf-8")
            if not data:
                continue
            req = json.loads(data)
            txt = req["text"].strip()
            out = req["out"]
            if txt and txt[-1] not in ".!?":
                txt += "."
            ref_txt = ref_txt_file.read_text(encoding="utf-8").strip()
            f5.infer(
                ref_file=str(ref_wav),
                ref_text=ref_txt,
                gen_text=txt,
                file_wave=out,
                nfe_step=12,
                cfg_strength=2.0,
                sway_sampling_coef=-1.0,
                speed=1.0,
                seed=42,
            )
            conn.sendall(b"OK")
        except Exception as e:
            conn.sendall(f"ERR:{e}".encode("utf-8"))
        finally:
            conn.close()

if len(sys.argv) > 1 and sys.argv[1] == "--daemon":
    run_server()
    sys.exit(0)

if len(sys.argv) < 2:
    print("Usage:\n  tts_julio \"Text to speak\"\n  tts_julio \"Text to speak\" /path/to/export.wav\n  tts_julio \"Text to speak\" /path/to/export.mp3")
    sys.exit(1)

last_arg = sys.argv[-1].lower()
export_requested = len(sys.argv) >= 3 and last_arg.endswith((".wav", ".mp3", ".ogg", ".flac"))

if export_requested:
    texto = " ".join(sys.argv[1:-1]).strip()
    final_export_path = Path(sys.argv[-1]).expanduser().resolve()
else:
    texto = " ".join(sys.argv[1:]).strip()
    final_export_path = default_out

final_export_path.parent.mkdir(parents=True, exist_ok=True)
tmp_wav_path = final_export_path if final_export_path.suffix.lower() == ".wav" else esc / "_tmp_tts_render.wav"

def send_req():
    c = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
    c.connect(SOCK)
    c.sendall(json.dumps({"text": texto, "out": str(tmp_wav_path)}).encode("utf-8"))
    return c.recv(1024).decode("utf-8")

try:
    res = send_req()
except Exception:
    print("⚡ Starting resident TTS engine in RAM (only takes a moment on first run)...")
    subprocess.Popen([sys.executable, __file__, "--daemon"], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, start_new_session=True)
    for _ in range(90):
        time.sleep(0.5)
        if os.path.exists(SOCK):
            try:
                res = send_req()
                break
            except Exception:
                pass
    else:
        raise RuntimeError("Timed out waiting for the TTS daemon to start.")

if res == "OK":
    if final_export_path.suffix.lower() != ".wav":
        subprocess.run(["ffmpeg", "-y", "-i", str(tmp_wav_path), str(final_export_path)], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=True)
        tmp_wav_path.unlink(missing_ok=True)
    print(f"💾 Audio exported to: {final_export_path}")
    print("🔊 Playing...")
    play_audio(final_export_path)
else:
    print("❌ Error:", res)
EOF
chmod +x ~/.local/bin/tts_clon.py

6. Register the Command in Your Shell (CachyOS & Linux Mint)

On CachyOS (fish shell)

Create and save the tts_julio function permanently:

Fragmento de código
function tts_julio
    python3 ~/.local/bin/tts_clon.py $argv
end
funcsave tts_julio

On Linux Mint (bash shell)

Add the alias to your ~/.bashrc and reload it:

Bash
echo 'alias tts_julio="python3 ~/.local/bin/tts_clon.py"' >> ~/.bashrc
source ~/.bashrc

7. How to Use and Export Your Cloned Voice

You can now run your cloned voice from any directory on your system:

  1. Speak immediately (saves by default to tts_salida.wav on your Desktop):

    Fragmento de código
    tts_julio "Hola amigos, esta es una prueba de mi voz clonada corriendo localmente en Linux."
    
  2. Export the generated voice to a custom .wav file (for Ardour, Kdenlive, or automation scripts):

    Fragmento de código
    tts_julio "Bienvenidos a un nuevo video del canal." ~/Escritorio/intro_video.wav
    
  3. Export directly to .mp3 or .ogg: If the target path ends in .mp3, .ogg, or .flac, the script automatically converts the output via ffmpeg:

    Fragmento de código
    tts_julio "El respaldo de la base de datos ha finalizado correctamente." ~/Escritorio/alerta_sistema.mp3
    
  4. Free up 5.4 GB of Hugging Face cache (Optional): Once f5_es_fast.safetensors has been generated on your Desktop, you can safely remove the original 5.4 GB .pt file from your Hugging Face cache:

    Fragmento de código
    rm -rf ~/.cache/huggingface/hub/models--andromeda0302--E2-F5-TTS-SPANISH

Comments

Popular posts from this blog

🛡️ The Ultimate Guide and Technical Manual for CyberBackup (Bender Rodríguez Edition): The Only Backup System That Respects Humans... In Its Own Way

🚀 Automating and Controlling SAP Vouchers with CodeIgniter 4, ODBC, and Service Layer: The Ultimate Guide & SapVouchersController Module

Updates in Framework Codeigniter 4.4.4 to 4.5.2

📷 How to Install V380 Pro on Linux Mint (Step by Step)

🚀 How to Automate TikTok Shorts Uploads from Linux: Playwright, Local AI (Ollama), and Systemd Without Getting Banned

🤠⚡ Deconstructing MySQL Error 150: How a Corrupted Dump Turned into a Cyberpunk SQL Injector