Varnish VCL compilation failed: reload rejected and the old VCL still live

You deployed a new VCL file, ran systemctl reload varnish or varnishadm vcl.load, and the reload was rejected. The VCC-compiler printed an error with a line and column number, and the management process refused to load the new configuration. The previous VCL is still serving traffic.

This is not an outage. Varnish reloads are hitless: the new VCL is compiled and loaded into memory before activation. When compilation fails, the old VCL stays in place and no client sees an interruption. But your change did not land. Until you fix the compilation error and retry, the running configuration is whatever was active before the failed deploy.

What this means

VCL is not interpreted at runtime. It is compiled to C, then compiled to a shared object, and loaded into the Varnish worker process. The management process (the root-owned varnishd) handles this pipeline. When you run vcl.load <name> <file> or trigger systemctl reload varnish (which calls the varnishreload script), the management process runs the VCC-compiler on your VCL file.

If the VCC-compiler finds a syntax error, type mismatch, or scope violation, it prints a message prefixed with Message from VCC-compiler: and exits. The CLI returns error code 106 with the message VCL compilation failed. The named VCL is never loaded into memory. The previously active VCL continues serving all traffic.

The hitless reload mechanism means a broken VCL cannot take down your service. The tradeoff is that a failed reload is easy to miss: traffic looks normal, dashboards are green, and the only evidence is in the CLI output or the systemd journal.

flowchart TD
    A["VCL reload rejected"] --> B["Read VCC-compiler error"]
    B --> C["Note line, column, subroutine"]
    C --> D["Fix VCL file"]
    D --> E["Test: varnishd -C -f file"]
    E -->|Compiles OK| F["Load: vcl.load name file"]
    E -->|Fails| D
    F -->|Success| G["Activate: vcl.use name"]
    F -->|Duplicate name| H["Use unique name"]
    H --> F
    G --> I["Verify: vcl.list"]
    I -->|New VCL active| J["Done"]
    I -->|Old VCL still active| B

Common causes

Quick checks

All read-only and safe on a production instance.

# Show all loaded VCLs with status and temperature
varnishadm vcl.list

# Test-compile a VCL file without affecting the running instance.
# Add -p vmod_path=/path/to/vmods if you use non-standard VMOD locations.
varnishd -C -f /etc/varnish/default.vcl

# Check VCL lifecycle counters (accumulated VCLs indicate loads without cleanup)
varnishstat -1 -f MAIN.n_vcl -f MAIN.n_vcl_avail -f MAIN.n_vcl_discard -f MAIN.vcl_fail

# Show the source of the currently active VCL
varnishadm vcl.show $(varnishadm vcl.list | awk '/active/ {print $NF; exit}')

# Review journal for recent reload errors
journalctl -u varnish --since "1 hour ago" --no-pager | grep -iE "vcl|VCC|compil"

# Check child process stability (a bad VCL can cause runtime panics after activation)
varnishstat -1 -f MGT.child_panic -f MGT.child_died -f MGT.child_start

How to diagnose it

1. Confirm the old VCL is still active. Run varnishadm vcl.list. The active status marks the VCL currently serving traffic. If your intended new VCL name shows available or is absent, the load failed or was never activated. Multiple available entries indicate previous successful loads that left old VCLs in memory without cleanup.

2. Read the compiler error. If you ran the reload through systemctl, check the journal:

   journalctl -u varnish --since "30 min ago" --no-pager | grep -A5 "VCC-compiler"
   

   The error includes the file path, line number, column, and the offending token. For scope violations, it names the variable and the subroutine where it was used. This is usually enough to identify the exact problem.

3. Reproduce with varnishd -C. Before making changes, confirm the error reproduces:

   varnishd -C -f /etc/varnish/default.vcl
   

   A non-zero exit and the same compiler error confirm you are looking at the right file. If varnishd -C succeeds, the file on disk differs from what the reload script passed to the management process. This happens with templated configurations or wrong file paths in the systemd unit.

4. Fix the VCL and re-test. Make the change, then run varnishd -C again until compilation succeeds.

5. Load the fixed VCL with a unique name:

   varnishadm vcl.load newconfig_$(date +%s) /etc/varnish/default.vcl
   

   If the name already exists, you get exit code 106 with Already a VCL named .... Pick a different name or discard the old one first with vcl.discard.

6. Activate the new VCL:

   varnishadm vcl.use newconfig_1234567890
   

   This is the hitless cutover. The old VCL moves to available status and the new one becomes active.

7. Verify and clean up:

   varnishadm vcl.list
   varnishadm vcl.discard oldconfig
   

   Confirm the new VCL shows active. Optionally discard the old one. You cannot discard the active VCL; make sure it is in available state first.

Metrics and signals to monitor

Fixes

Variable scope violations

The VCC-compiler enforces strict scope rules. resp.http.* headers can only be set in vcl_deliver or vcl_backend_response, not in vcl_recv. The error message names the variable and the subroutine. Move the operation to the correct subroutine, or use the appropriate variable for that scope (req.http.* in vcl_recv, beresp.http.* in vcl_backend_response).

VCL version declaration

The first line of every VCL file must declare the version: vcl 4.0; or vcl 4.1;. VCL 4.1 was introduced in Varnish 6.0. You cannot mix versions across include directives: a VCL 4.1 file cannot include a VCL 4.0 fragment. If you upgraded Varnish and are unsure which version to use, vcl 4.1; is valid on Varnish 6.0 and later. Without UDS backends, vcl 4.0; continues to work.

Duplicate VCL name

The vcl.load command requires a unique name. The varnishreload script generates timestamped names automatically, so this error typically appears when loading VCLs manually. Check vcl.list for existing names and either pick a new one or vcl.discard the old one first. A VCL must not be active to be discarded.

VMOD import failures

If an import directive references a VMOD that is not installed or is incompatible with your Varnish version, compilation fails at the import line. Verify the VMOD is installed in the correct module path for your Varnish version. In Varnish 3.x, directors were built in using the director keyword. In VCL 4.0+, the old director keyword does not work and you must use import directors; with the VMOD API instead.

vcl.inline heredoc syntax

When using vcl.inline from the CLI with a heredoc, the shell must deliver the full VCL text as an argument:

varnishadm vcl.inline test << EOF
vcl 4.1;
...
EOF

Standard bash accepts both <<EOF and << EOF; the spacing itself is not the cause. The failure mode typically involves how the CLI client handles multi-line input or special characters in the VCL text.

If the heredoc content does not reach vcl.inline correctly, the VCL is interpreted as empty and produces VCL version declaration missing.

Working directory deleted

If the Varnish working directory (specified with -n) has been deleted, vcl.inline can fail with a generic VCL compilation failed message because Varnish cannot create the temporary VCL directory. Later versions include errno/strerror in the error output, but on older builds the message gives no hint about the filesystem problem. Check that the working directory exists and is writable by the Varnish user.

systemd reload can kill varnishd

On some older systemd versions, systemctl reload varnish with a bad VCL can kill the varnishd process entirely (reported with systemd 239 on CentOS 8). The workaround is to prefix the ExecReload command in the systemd unit with a dash so systemd treats a non-zero reload exit as non-fatal:

ExecReload=-/opt/varnish/sbin/varnishreload -m 3 -w 10

Note: the exact path to varnishreload and the behavior of its -m and -w flags vary by distribution and Varnish version; consult your distribution’s package documentation.

The leading dash tells systemd to ignore the exit code. Check your systemd unit file and update it if it lacks this prefix.

Prevention

Pre-compile before deploying. Run varnishd -C -f <file> in your CI pipeline or deploy script. A non-zero exit stops the deploy before it reaches the running instance. This catches syntax errors, scope violations, and missing VMOD imports without touching production.

Use unique VCL names for manual loads. If you load VCLs manually rather than through varnishreload, include a timestamp or commit hash in the name to avoid duplicate-name errors.

Monitor MAIN.n_vcl. A growing count without corresponding n_vcl_discard increases means successful loads are accumulating VCLs in memory. Each loaded VCL consumes resources even when not active.

Check vcl.list after every deploy. Confirm the intended VCL is active. This takes seconds and catches silent reload failures before they become incidents.

Keep VCL version declarations consistent. All included files must use the same VCL version as the parent file. Mixing vcl 4.0; and vcl 4.1; across includes produces a compilation error.

How Netdata helps

Netdata’s per-second metric resolution lets you see the exact moment a VCL activation takes effect. Correlate step changes in the metrics below with your deploy timestamp to confirm the new configuration is behaving as expected.

• MAIN.n_vcl: a step increase without a matching n_vcl_discard increase means old VCLs are accumulating from manual loads without cleanup.
• MAIN.vcl_fail: any nonzero rate after activation indicates a VCL that compiled cleanly but fails during request processing.
• MGT.child_panic / MGT.child_died: increments immediately after a VCL change signal that the new configuration is crashing the child process.
• Cache hit rate: an unexpected drop after a successful reload may indicate a logic error that passed compilation but breaks caching behavior.

Related guides

• Varnish Error 503 Backend fetch failed: what the error page actually means
• Varnish backend_fail, backend_unhealthy, and backend_busy: three different backend problems
• Varnish backend connection reuse low: keepalive not working and slow TTFB
• Varnish backend probe configuration: threshold, window, interval, and initial
• Varnish backend is sick: health probes, all-backends-sick, and grace
• Varnish ban list growing: O(n) lookups and the lurker falling behind
• Varnish ban lurker not keeping up: contention and ban_lurker_sleep
• Varnish cache hit ratio dropped: hit rate collapse and backend overload
• Varnish cache stampede: a popular object expires and the herd hits the backend
• Varnish ESI errors: broken pages and workspace pressure from Edge Side Includes
• Varnish fetch_failed: backend connected but the fetch broke
• Varnish grace masking a backend outage: the ticking-clock incident