Back to all posts

Tuning qBittorrent and WireGuard Throughput on NixOS

nixosqbittorrenthomelabwireguard

My media server runs qBittorrent inside a WireGuard VPN namespace on NixOS, managed by nixarr. Nixarr handles the heavy lifting — VPN namespace, service wiring, the *arr stack — but there’s still room to squeeze more throughput out of the setup. Here’s what I tuned at the kernel, qBittorrent, and systemd level.

BBR Congestion Control

Linux defaults to CUBIC congestion control, which uses packet loss as its congestion signal — when it sees a dropped packet, it backs off hard. Over a WireGuard tunnel, you get occasional packet loss from the VPN path itself (encapsulation overhead, server load) that isn’t actual network congestion. CUBIC can’t tell the difference and throttles anyway.

BBR (Bottleneck Bandwidth and Round-trip propagation time) takes a different approach: instead of reacting to loss, it directly models the bottleneck bandwidth and RTT. This means it doesn’t back off from non-congestion loss, fills the pipe faster, and stays there.

boot.kernel.sysctl = {
  "net.core.default_qdisc" = "fq";
  "net.ipv4.tcp_congestion_control" = "bbr";
};

BBR requires fq (fair queueing) as the qdisc — it won’t pace correctly with the default fq_codel.

TCP Buffer Sizing

To keep a link fully utilized, the kernel needs enough buffer space to cover all the data “in flight” between sender and receiver. This is the bandwidth-delay product: with 1Gbps throughput and ~30ms RTT through the WireGuard tunnel, that’s roughly 3.75MB of data in transit at any moment.

The default Linux rmem_max is ~212KB. TCP autotuning can never grow the receive window beyond that, so the link is capped well below its actual capacity. Raising the maximum to 16MB lifts that ceiling:

boot.kernel.sysctl = {
  "net.core.rmem_max" = 16777216;
  "net.core.wmem_max" = 16777216;
  "net.ipv4.tcp_rmem" = "4096 87380 16777216";  # min default max
  "net.ipv4.tcp_wmem" = "4096 65536 16777216";
};

The three values in tcp_rmem/tcp_wmem are minimum, default, and maximum. The kernel auto-tunes within this range per connection — setting a higher maximum doesn’t allocate 16MB per socket, it just allows the window to grow that large when needed.

Preventing Slow Start After Idle

TCP normally resets its congestion window when a connection goes idle. With hundreds of torrent peer connections going active and inactive, this means constantly re-probing bandwidth:

boot.kernel.sysctl = {
  "net.ipv4.tcp_slow_start_after_idle" = 0;
};

Disabling this lets connections pick up where they left off instead of starting from scratch.

MTU Probing

WireGuard reduces the effective MTU to ~1420 bytes due to encapsulation overhead. MTU probing lets TCP discover the optimal segment size through the tunnel instead of relying on static configuration:

boot.kernel.sysctl = {
  "net.ipv4.tcp_mtu_probing" = 1;
};

qBittorrent Disk I/O Tuning

Getting data to disk fast matters when seeding. By default, qBittorrent uses a single async I/O thread and conservative caching. On NixOS with nixarr, these settings go in extraConfig:

nixarr.qbittorrent.extraConfig = {
  BitTorrent = {
    "Session\\AsyncIOThreads" = 8;
    "Session\\HashingThreads" = 2;
    "Session\\CheckingMemUsage" = 256;       # MB for hash checking
    "Session\\DiskIOReadMode" = 0;            # 0 = enable OS cache
    "Session\\DiskIOWriteMode" = 0;           # 0 = enable OS cache
    "Session\\CoalesceReadsAndWrites" = true;
  };
};
  • AsyncIOThreads: More threads for parallel disk operations. 8 is reasonable for an SSD.
  • HashingThreads: Pieces need to be hashed after download. 2 threads keeps up without stealing too much CPU.
  • DiskIOReadMode/WriteMode 0: Uses the OS page cache rather than qBittorrent’s internal cache. The kernel’s cache is almost always smarter.
  • CoalesceReadsAndWrites: Batches adjacent disk operations together, reducing syscall overhead.

qBittorrent Network Tuning

For the send buffer and connection settings:

nixarr.qbittorrent.extraConfig = {
  BitTorrent = {
    # Connection limits
    "Session\\MaxConnections" = 500;
    "Session\\MaxConnectionsPerTorrent" = 100;
    "Session\\MaxUploads" = 50;
    "Session\\MaxUploadsPerTorrent" = 8;

    # Send buffer tuning
    "Session\\SendBufferWatermark" = 5120;        # 5MB send buffer
    "Session\\SendBufferLowWatermark" = 1024;
    "Session\\SendBufferWatermarkFactor" = 250;
    "Session\\SocketBacklogSize" = 1000;

    # Choking algorithm
    "Session\\ChokingAlgorithm" = 0;              # Fixed slots
    "Session\\SeedChokingAlgorithm" = 1;          # Fastest upload
  };
};

The choking algorithm determines how upload slots are distributed among peers. “Fixed slots” with “Fastest upload” prioritizes peers that can actually move data, which maximizes upload throughput.

Systemd Priority Scheduling

Jellyfin streams need to be responsive. Background services like qBittorrent and the *arr stack shouldn’t cause buffering:

# qBittorrent: lowest priority
systemd.services.qbittorrent.serviceConfig.Nice = 15;
systemd.services.qbittorrent.serviceConfig.IOSchedulingClass = "idle";

# *arr services: low but not idle
systemd.services.radarr.serviceConfig.Nice = 10;
systemd.services.sonarr.serviceConfig.Nice = 10;
systemd.services.prowlarr.serviceConfig.Nice = 10;
systemd.services.bazarr.serviceConfig.Nice = 10;

Nice = 15 deprioritizes CPU scheduling. IOSchedulingClass = "idle" means qBittorrent only gets disk I/O when nothing else wants it. The *arr services get Nice = 10 — lower than default but higher than qBittorrent, since they need to process releases promptly.

Binding qBittorrent to the VPN

One more important piece: qBittorrent must restart when the VPN restarts, and it should never run without VPN:

# Restart qBittorrent when VPN restarts
systemd.services.qbittorrent.bindsTo = [ "wg.service" ];

# NixOS assertions - fail the build if VPN is disabled
assertions = [
  {
    assertion = config.nixarr.qbittorrent.enable
      -> config.nixarr.qbittorrent.vpn.enable;
    message = "qBittorrent MUST have VPN enabled";
  }
];

bindsTo is stronger than requires — if the WireGuard service stops or restarts for any reason (like a VPN failover), qBittorrent stops too and comes back up on the new tunnel.

The Unsolved Problem: Jellyfin Background Tasks

One thing I haven’t figured out yet is prioritizing Jellyfin’s background work separately from its streaming. Jellyfin runs intro detection, library scanning, and thumbnail generation — all CPU and I/O intensive — under the same systemd service as real-time transcoding.

You can’t just Nice the whole Jellyfin service because that would also degrade streaming performance. What you’d really want is for background tasks (scheduled scans, intro detection) to run at low priority while keeping transcoding at normal priority. But Jellyfin doesn’t separate these into different processes.

For now, scheduling heavy tasks to run at night and limiting the intro detection plugin’s concurrency is the pragmatic workaround. A proper solution might involve a wrapper around ffmpeg that detects background task arguments and applies nice/ionice selectively — but that’s fragile and I haven’t needed it badly enough to build it yet.

Comments