Tomcat sessions eating the heap: bots, getSession(true), and unbounded growth

Post-GC heap baseline climbing steadily, Full GC frequency increasing, and eventually an OutOfMemoryError or a GC death spiral that leaves the JVM effectively unresponsive. Thread dumps look normal, CPU is dominated by GC threads, and a restart clears the problem temporarily before the cycle repeats within hours or days.

A common and overlooked root cause is HTTP session accumulation. Under Tomcat’s default StandardManager, every active session lives in JVM heap. When sessions are created faster than they expire, they fill the heap, promote to Old Gen, and drive major GCs. Sessions are frequently the largest heap consumer in a Tomcat app, and unbounded session count is a classic memory leak vector.

The specific pattern covered here is crawler traffic combined with application code that calls getSession(true). Bots that do not send cookies never join a session. Each request creates a new session, the Set-Cookie header is ignored, and the next request starts the cycle again. With the default session timeout of 30 minutes and maxActiveSessions at -1 (unlimited), a modest crawler storm can create hundreds of thousands of live sessions, each consuming heap.

What this means

Tomcat’s default session manager, StandardManager, stores all session state in memory. There is no eviction beyond timeout-based expiry. Sessions persist until they time out (default 30 minutes in the global web.xml) or are explicitly invalidated. The maxActiveSessions attribute defaults to -1, meaning the manager imposes no upper bound on live sessions.

The Servlet API contract for getSession(true) is the key mechanic. If no session exists for the request, a new one is created and a Set-Cookie header is sent back. If the client never returns that cookie, the session is never rejoined. The next request from that client creates yet another new session. The Servlet API documents this explicitly: if the client chooses not to join the session, getSession returns a different session on each request.

This is documented behavior, not a bug. But when the clients are bots, scrapers, monitoring agents, or misconfigured HTTP clients that drop cookies, and the application calls getSession(true) (or bare getSession(), which defaults to create=true) on every request path, the result is unbounded session creation.

flowchart TD
    A[Bot request, no cookie] --> B[App calls getSession true]
    B --> C[New session created in heap]
    C --> D[Set-Cookie sent, bot ignores it]
    D --> E[Next request: no cookie]
    E --> B
    C --> F[Sessions accumulate]
    F --> G[Promote to Old Gen]
    G --> H[Full GC frequency rises]
    H --> I[Post-GC baseline climbs]
    I --> J[OOM or GC death spiral]

JSPs make this worse. Unless a JSP page declares <%@page session="false"%>, the JSP engine implicitly calls getSession() on every request. A site serving JSP pages to crawlers without that directive will create a session for every bot hit, even for a page that renders static HTML and never touches session data.

Common causes

Quick checks

These are read-only and safe on a production instance. The JMX and jstack calls attach to the JVM; they do not mutate state.

# Active session count via JMX for the ROOT context.
# Requires JMX enabled on the JVM (com.sun.management.jmxremote.*).
# Port 9090 is an example; use whatever your CATALINA_OPTS sets.
java -jar jmxterm.jar -l localhost:9090 -n -v silent -e \
  "get -b Catalina:type=Manager,host=localhost,context=/ activeSessions sessionCounter expiredSessions rejectedSessions"

# Old Gen usage and GC count over 5 seconds.
# If multiple Tomcat JVMs run on the host, pgrep returns several PIDs;
# pick the right one explicitly instead of command substitution.
jstat -gcutil <pid> 1000 5

# Count bot/crawler requests in today's access log.
# Path and extension vary by distribution; adjust to your server.xml AccessLogValve config.
grep -iE 'bot|slurp|crawler|spider' /var/log/tomcat/localhost_access_log.$(date +%Y-%m-%d).txt | wc -l

# Top User-Agents by request count.
# The awk field index assumes the combined log format; verify against your pattern.
awk -F'"' '{print $6}' /var/log/tomcat/localhost_access_log.$(date +%Y-%m-%d).txt | \
  sort | uniq -c | sort -rn | head -20

# Configured session timeout (global default).
grep -A2 'session-config' $CATALINA_BASE/conf/web.xml

# maxActiveSessions setting. Absent means default -1, unlimited.
grep -rn 'maxActiveSessions' $CATALINA_BASE/conf/ 2>/dev/null

The ratio that matters is sessionCounter (cumulative sessions ever created) versus expiredSessions (cumulative sessions expired). If sessionCounter is climbing much faster than expiredSessions, sessions are being created faster than the 30-minute timeout can reap them.

activeSessions is not reliably exposed via Manager Status XML across all Tomcat versions. JMX is the authoritative source. The Manager HTML status page shows session counts, but for scripted monitoring use the JMX bean Catalina:type=Manager,host=localhost,context=/<app>.

How to diagnose it

1. Confirm activeSessions is growing monotonically. Sample the JMX attribute every minute. If it never plateaus during a traffic window, sessions are accumulating. A drop that lags falling traffic by roughly the session-timeout interval is normal, because sessions persist until timeout.

2. Correlate the growth with access log bot traffic. Pull the top User-Agents from the access log for the same window where activeSessions is climbing. A surge of requests from Googlebot, Bingbot, semrush, Ahrefs, or generic scrapers that tracks the session growth curve is the signature.

3. Estimate the session memory footprint. Multiply activeSessions by an estimate of per-session size. Per-session size varies enormously by application, from roughly 1KB for minimal sessions to 1MB or more for sessions holding large object graphs. A reasonable operating budget is that session count times per-session size should not exceed about 30% of max heap. If you do not know your per-session size, capture a heap dump and measure it directly.

4. Confirm Old Gen pressure tracks session growth. Watch the Old Gen memory pool post-GC. If it rises in step with activeSessions, sessions are the dominant heap consumer and are promoting into Old Gen where they drive Full GCs.

5. Audit the application for getSession calls. Search the codebase for getSession(true), getSession(false), and bare getSession(). Any call path reachable from a stateless request (a page that does not need session state) is a candidate for removal. Check JSPs for the absence of session="false".

Metrics and signals to monitor

Fixes

Install CrawlerSessionManagerValve

Tomcat ships a valve for this problem: CrawlerSessionManagerValve. It detects requests from known crawler User-Agents and associates all of them with a single session per client identifier, regardless of whether they send a cookie. This collapses the per-request session explosion into one session per bot.

The valve is added to a Host or Context in server.xml or context.xml. It must be explicitly configured; it is not in the default pipeline. If you also use RemoteIpValve, place CrawlerSessionManagerValve after it so the valve sees the resolved client IP.

<Valve className="org.apache.catalina.valves.CrawlerSessionManagerValve"
       crawlerUserAgents=".*[bB]ot.*|.*Yahoo! Slurp.*|.*Feedfetcher-Google.*"
       sessionInactiveInterval="60"/>

The default crawlerUserAgents regex catches many common bots but will miss crawlers that do not match. Tune the regex against your actual bot traffic pulled from the access log. The sessionInactiveInterval governs how long the crawler-to-session mapping is cached.

Tradeoff: this only helps if the bot matches your regex. Sophisticated scrapers that forge browser User-Agents will not be caught, and you may need to add crawlerIps for known datacenter ranges.

Reduce session timeout

The global default is 30 minutes. If your application does not need long-lived sessions, reducing this to 5 or 10 minutes directly limits how many sessions can accumulate before the reaper clears them.

<session-config>
  <session-timeout>10</session-timeout>
</session-config>

Tradeoff: shorter timeouts log out idle users sooner. Measure the impact on your real session duration distribution before cutting aggressively.

Set maxActiveSessions

Setting maxActiveSessions to a finite value caps the damage. The failure mode is abrupt: when the limit is reached, any attempt to create a new session throws IllegalStateException, which surfaces as a 500 error to the user. This is a circuit breaker, not a graceful throttle. Use it as a backstop, not as the primary control.

Tradeoff: legitimate users may see 500s during a bot storm. Pair this with CrawlerSessionManagerValve so bots rarely consume slots.

Disable session creation in JSPs

Add <%@page session="false"%> to JSP pages that do not need session access. This is especially important for landing pages, error pages, and any content frequently crawled. Without this directive, the JSP engine calls getSession() implicitly on every request, creating a session even for a page that renders static HTML.

Audit and remove unnecessary getSession(true) calls

Search the codebase for getSession(true) and bare getSession() calls on request paths that do not need session state. Replace with getSession(false) where the code only needs to read existing session data, or remove the call entirely. This is the root fix for applications that create sessions as a side effect of touching the request object.

Consider PersistentManager

PersistentManager can swap idle sessions to disk, reducing heap pressure. This is a heavier change and introduces disk I/O and serialization overhead. It is appropriate when sessions are legitimately large and numerous, not as a workaround for a bot-driven leak. It does not solve the underlying creation-rate problem, only the residency problem.

Prevention

• Monitor activeSessions per context as a first-class metric. Alert on monotonic growth that does not plateau. The signal requires JMX; it is not reliably exposed via Manager Status XML.

• Track sessionCounter and expiredSessions deltas. If creation consistently outpaces expiry, investigate before it becomes an OOM at 3 a.m.

• Watch the correlation between bot traffic and session creation. A sudden bot spike on a site without CrawlerSessionManagerValve is a leading indicator.

• Budget session memory explicitly. Estimate activeSessions peak times per-session size against max heap. If that number exceeds roughly 30% of heap, you are operating without margin.

• Capture a heap dump before restarting during an incident. Once the JVM restarts, the evidence is gone. See the heap dump capture guide in Related guides.

How Netdata helps

• Per-second activeSessions collection via JMX shows session growth as it happens, not on a minute lag. Correlating the activeSessions curve with the Old Gen post-GC baseline on the same dashboard confirms whether sessions are the heap consumer rather than a cache or application leak.

• Anomaly detection on session count flags monotonic growth that a static threshold would miss. A session count that is normal at peak traffic may be a leak at off-peak hours; the detection adapts to the diurnal pattern rather than firing false positives.

• Correlation of access log bot traffic with session creation shortens diagnosis from “heap is full” to “this specific crawler spike created 80,000 sessions.” Surfacing request rate by User-Agent alongside JVM metrics pinpoints the source.

• GC frequency and pause duration alongside heap pools lets you watch the spiral form in real time and confirm the fix (CrawlerSessionManagerValve, shorter timeout) actually flattened the curve.

• Post-GC heap baseline trending distinguishes a real leak from normal sawtooth. The valleys, not the peaks, indicate retained live data.

Related guides

• Tomcat 5xx error rate: separating server failures from crawler 404s
• Tomcat frequent Full GC: pause time, G1, and the 5% overhead rule
• Tomcat GC death spiral: full GCs dominating and throughput collapsing
• Tomcat heap dump before restart: capturing evidence with jmap and jstack
• Tomcat heap usage: watch the post-GC baseline, not the sawtooth peak
• Tomcat classloader leak on redeploy: why the old WebappClassLoader never dies