Monitor AWS infrastructure through Amazon CloudWatch. This collector discovers CloudWatch metrics for a curated set of AWS services and renders them as Netdata charts, with minimal configuration.
This collector queries runtime metrics from Amazon CloudWatch. It complements the
AWS EC2 Compute instances
integration, which exposes EC2 inventory and capacity information, and the
AWS Quota integration, which exposes
AWS Service Quotas. These integrations use different AWS data sources and do not replace one another.
CloudWatch coverage is defined by one or more profiles – YAML files declaring a CloudWatch namespace, an exact resource-dimension grain, supported regions, metrics, statistics, and chart template. A service can use multiple profiles when AWS publishes distinct grains, and coverage can be extended without collector code changes. See the AWS CloudWatch profile format for the complete schema and authoring rules.
Tip - Need a service that isn’t listed?
Request a profile – it’s just a YAML file, no code change. Open a feature request and attach the service’s CloudWatch metric schema, captured with this read-only command. It prints only metric and dimension names (no resource IDs, ARNs, or metric values), so the output is safe to share:
Replace AWS/<Service> with the service namespace (for example AWS/AmazonMQ) and <your-region> with a Region where the service runs. The exact metrics and dimensions in the output are what we need to author a correct profile quickly.
The collector compiles named credential sources, monitored targets, and ordered collection rules into an immutable runtime plan. Profiles with identifying dimensions discover available metrics with one CloudWatch ListMetrics scan per target, region, and namespace, using the least restrictive recent-activity policy required by the participating selected series, then apply each profile’s exact dimension matcher. A profile whose dimensions are all constants compiles a known static instance and skips ListMetrics. Optional resource-tag predicates are resolved with the Resource Groups Tagging API before query expansion. Each selected series has a resolved query policy: aggregation period, rolling lookback, and publication delay. GetMetricData searches the aligned rolling window for the newest complete finite datapoint, while Netdata receives the retained numeric value on every collection cycle. Each target resolves its AWS account id through sts:GetCallerIdentity; target names remain distinct execution identities even when they resolve to the same account. Rule order, then target order within each rule, resolves overlap: the first matching rule/target owns each overlapping exported metric/statistic series.
This collector is supported on all platforms.
This collector supports collecting metrics from multiple instances of this integration, including remote instances.
Every target requires cloudwatch:GetMetricData. It also requires cloudwatch:ListMetrics when any selected profile has an identifying dimension and therefore needs discovery; an all-constant profile such as billing_total is queried directly. The collector calls sts:GetCallerIdentity for account attribution, but AWS does not require an explicit permission grant for that operation. A credential source used by a target with assume_role additionally requires sts:AssumeRole for that role ARN. Resource tag filtering (rule_defaults.filters.resource_tags or rules[].filters.resource_tags) and resource tag labels (labels.resource_tags) additionally require tag:GetResources.
Default Behavior
Auto-Detection
A rule that omits profiles selects all default-enabled profiles for its targets and regions. A rule that omits metrics collects every metric exported by those profiles. When present, metrics groups exact AWS MetricNames by profile; group statistics are inherited by included metrics unless a metric supplies a replacement list. The collector emits charts only for profiles with live metrics. Discovery and the query blueprint are cached; discovery refreshes every discovery.refresh_every seconds (default 300).
Limits
Minimum collection interval is 60 seconds (CloudWatch’s minimum metric period).
Query timing resolves field-by-field from rules[].query, rule_defaults.query, nested metric query defaults, nested profile query defaults, and the built-in 10-minute publication delay. The stock daily S3 storage profile uses a conservative one-day collector policy. AWS documents that S3 storage metrics are reported once per day, but does not guarantee publication within one day.
A successful sparse query can replay its newest eligible CloudWatch value for up to lookback. During transient AWS failures, the retained value can be replayed longer, until a successful query replaces or expires it.
Query-plan preflight rejects more than 20,000 selected series, 600,000 all-due datapoints, or 40 packed GetMetricData requests before allocating AWS query structures. Billing units of up to five statistics for one structural metric are kept in one request.
limits.max_instances defaults to 1000 distinct final static or discovered instances after tag filtering and overlap resolution. Exceeding it rejects the refreshed query plan; instances are never silently truncated.
limits.max_discovery_groups defaults to 64 unique (target, region, namespace) groups per job. Compatible rules and profiles share a group. The safeguard catches accidental expansion and can be raised to the hard maximum of 100; larger collection must be split across jobs because one refresh can admit at most 100 groups that reach ListMetrics.
Each discovery group is independently bounded to 100 ListMetrics pages, 50,000 scanned metrics, 1,000,000 residual same-shape profile matches, and 20,000 candidate instances. Overflow fails the group without replacing its previous snapshot.
One discovery refresh is additionally bounded to 100 admitted ListMetrics SDK operations, 50,000 scanned metrics, 1,000,000 residual profile matches, 20,000 retained candidates, 64 MiB of conservatively weighted candidate storage, and one shared timeout. Every non-skipped group that resolves a client runs its first admitted operation before continuations share the remaining budget; skipped groups and client-resolution failures consume no operation budget. Successful replacements and failed-group carry-forward are rechecked as one bounded effective snapshot before installation. The AWS SDK can retry each admitted operation up to five wire attempts.
Aggregate discovery-limit, merged effective-snapshot limit, or timeout exhaustion discards the attempted refresh atomically. An existing snapshot remains active and discovery retries after discovery.refresh_every. On the first pass, any executable all-constant static profiles continue while dynamic discovery waits for its retry; without static work, total failure makes the collection attempt fail. Parent cancellation always aborts without advancing state.
Resources are labeled by their identifying CloudWatch dimensions (for example EC2 instance_id). Selected resource tags can additionally be attached as non-identity labels via labels.resource_tags; changing those tags updates labels without changing chart identity. (A dimension that is constant across resources, such as CloudFront’s Region=Global, is used to match and query metrics but is not turned into a label.)
Performance Impact
AWS bills CloudWatch API usage. GetMetricData is the cost driver; ListMetrics discovery falls under the free tier and then costs a fraction as much. As a rough anchor, GetMetricData is billed at roughly $0.01 per 1,000 metrics requested – confirm current CloudWatch pricing for your region. The collector sends one query per selected metric/statistic series, but for AWS billing up to five statistics requested for the same metric count as one metric request. Point-aware batching keeps each such five-statistic billing unit in one request. In normal operation each series is queried once per newly eligible effective-period window, not once per Netdata collection cycle. A transient request or result failure retries that series after one update_every; subsequent delays double within the same eligible window and are capped at its effective period. Those retries are also billable, so update_every affects failure-time cost even though it does not set normal query cadence. Cost otherwise scales with selected targets, instances, metrics, statistics beyond AWS’s grouping, periods, and lookback response work. Longer lookbacks increase requested datapoints and can disable CloudWatch’s three-hour recently-active discovery filter. The collector minimizes work with curated profiles, exact metric and resource-tag selection, exact dimension matching, single-statistic defaults, completed-sibling isolation, bounded retry backoff, billing-group-preserving batches, shared compatible discovery scans, cached discovery/query plans, and recently_active_only. To reduce cost further, narrow rules[].targets, rules[].profiles, rules[].metrics, rules[].regions, or configure resource tag filters.
The opt-in Billing profiles use a 10-minute period, so each selected single-statistic Billing series normally produces 144 billable metric requests per day before retries. For example, 200 selected Billing series produce 28,800 metric requests per day. Their 24-hour lookback reserves 144 datapoint slots per query (28,800 slots when all 200 are due), but AWS charges GetMetricData by metrics requested, not by those reserved datapoint slots. The total profile is static and performs no ListMetrics; the three dynamic Billing grains share one namespace discovery stream per target and refresh interval before pagination and SDK retries. Billing cardinality grows with services, linked accounts, and observed account/service pairs, so select only the grains you need.
Setup
You can configure the cloudwatch collector in two ways:
Create an AWS IAM identity with CloudWatch read access
The collector needs an IAM identity (user or role) allowed to read CloudWatch metrics. It resolves the AWS account identity with GetCallerIdentity, which does not require an explicit permission grant.
cloudwatch:ListMetrics and cloudwatch:GetMetricData do not support resource-level permissions, so "Resource": "*" is required – this is already least-privilege for these read actions. A job that selects only profiles whose dimensions are all constants, such as billing_total, can omit cloudwatch:ListMetrics; every dynamic profile needs it. GetCallerIdentity needs no explicit grant. Scope sts:AssumeRole to the specific role ARN(s) rather than *. To enable resource tag filtering or labels, also grant tag:GetResources (it likewise requires "Resource": "*").
Define one or more named credential sources:
default – the AWS SDK default credential chain (environment variables, shared config/credentials files, EC2 instance profile, or EKS IRSA). Recommended when Netdata runs inside AWS.
static – an explicit access key ID and secret access key, plus an optional session token. Use go.d secret references rather than plaintext values.
A target can use either source directly or use it to assume one IAM role. If the role trust policy requires an external ID, the role owner supplies that value; it is not an AWS password or access key. See AWS guidance for third-party access.
Enable CloudWatch Billing metrics before selecting Billing profiles
The Billing profiles are opt-in. Before collecting them, enable Receive CloudWatch Billing Alerts in AWS Billing Preferences as described in AWS’s Billing alarm documentation.
This setup action is separate from the collector’s runtime IAM policy:
Enabling the preference requires the account root user or an IAM principal allowed to view Billing information. The collector identity still needs only the CloudWatch read permissions described above.
Once enabled, AWS says Billing metric data collection cannot be disabled. Deleting CloudWatch alarms does not disable the metric feed.
The first data normally appears about 15 minutes after first enablement. AWS then calculates and publishes estimates several times daily, so a working job can legitimately have no chart or can hold the last published value between updates.
Billing data is published only in us-east-1, represents worldwide charges, and is reported only in USD. The charts show the latest published estimate for the current month, not a forecast.
For consolidated billing, enable the preference in the management/payer account. That account can expose the consolidated total plus linked-account views; a standalone/member view can contain fewer grains. If the management/payer account changes, enable the preference again in the new account.
AWS does not publish these Billing metrics for Amazon Partner Network (APN) accounts.
The collector’s Billing profiles use a 24-hour retrieval window and sample-and-hold the newest successful value. Around the UTC month boundary, a prior-month observation can remain visible until a successful query replaces or expires it; a transient AWS failure can retain it longer under the collector’s general replay policy. Treat the value as latest-published month-to-date data, not an invoice or real-time ledger.
Configuration
Options
The following options can be defined globally or per job.
A user profile file with the same basename as a stock profile overrides it.
Group
Option
Description
Default
Required
Collection
update_every
Data collection interval (seconds). Must be at least 60 (CloudWatch’s minimum period).
60
no
autodetection_retry
Recheck interval (seconds) when the job fails to start. Default 0 means no retry; set a positive value to keep retrying.
0
no
timeout
AWS operation timeout (seconds). Identity, resource-tag, and query operations use it for their operation scope; discovery shares one timeout across its whole refresh stage.
30
no
Authentication
credentials
List of named credential sources. Every source has a type of default (AWS SDK default chain) or static (explicit access/session credentials in type_static). Credential sources are reusable by targets and every defined source must be used.
Configuration used only when the credential source type is static.
no
credentials[].type_static.access_key_id
AWS access key ID. Required in type_static. Use a go.d secret reference such as ${env:AWS_ACCESS_KEY_ID}.
no
credentials[].type_static.secret_access_key
AWS secret access key. Required in type_static. Use a go.d secret reference; do not store plaintext credentials in the file.
no
credentials[].type_static.session_token
Optional AWS session token in type_static for temporary credentials. Use a go.d secret reference.
no
Targets
targets
Up to 64 named monitored AWS identities. A target uses one credential source directly or uses that source to assume one role. Targets remain distinct even when they resolve to the same AWS account.
yes
targets[].name
Target name referenced by collection rules.
yes
targets[].credentials
Name of the credential source used by this target.
yes
targets[].assume_role.role_arn
Optional IAM role ARN to assume using the target’s credential source.
no
targets[].assume_role.external_id
Optional value supplied by the role owner when the role trust policy requires an external ID. It is not an AWS password or access key.
no
Rules
rules
Ordered collection rules. Each rule selects targets, profiles, optional exact metrics, regions, and optional resource-tag filters. The earliest matching rule and target own each overlapping exported metric/statistic series.
yes
rules[].name
Unique rule name used in diagnostics.
yes
rules[].targets
Ordered names of monitored targets selected by this rule. Order breaks overlap ties within the rule.
yes
rules[].profiles.defaults
Include all default-enabled profiles. Defaults to true when profiles or defaults is omitted.
yes
no
rules[].profiles.include
Profile basenames to add explicitly. Set defaults to false to collect only this list, including profiles disabled by default. The higher-cardinality PrivateLink subnet view is privatelink_endpoint_subnet. Billing choices are billing_total, billing_service, billing_linked_account, and billing_linked_account_service.
no
rules[].profiles.exclude
Profile basenames to remove from the selected set. A profile cannot be both included and excluded.
no
rules[].metrics
Optional per-profile exact metric/statistic allowlists that narrow the profiles selected by this rule. Omit metrics to collect every metric exported by those profiles. The expanded rule supports at most 256 metric/statistic selections.
no
rules[].metrics[].profile
Profile basename. It must already be selected by rules[].profiles and may appear in only one metrics group per rule.
yes
rules[].metrics[].statistics
Optional non-empty default AWS statistics inherited by included metrics that omit their own list. Named statistics are case-insensitive.
no
rules[].metrics[].include
Non-empty list of exact, case-sensitive AWS CloudWatch MetricNames exported by this profile. Duplicate names are rejected.
yes
rules[].metrics[].include[].name
Exact, case-sensitive AWS CloudWatch MetricName exported by the profile.
yes
rules[].metrics[].include[].statistics
Optional non-empty replacement for the group statistics. Use Average, Minimum, Maximum, Sum, SampleCount, or p<N>; named statistics are case-insensitive. A metric with no replacement must inherit a group default.
no
rules[].regions
Canonical lowercase AWS region codes selected by this rule. The compiler intersects them with intrinsic profile restrictions; CloudFront and the Billing profiles support only us-east-1.
yes
Query Policy
rule_defaults.query
Shared query timing inherited field-by-field by collection rules. Omitted fields continue to profile or built-in fallbacks.
no
rule_defaults.query.period
Default CloudWatch aggregation period from 1m through 1d, as an exact multiple of 1m. An omitted rule period inherits this value before nested metric and profile query defaults.
no
rule_defaults.query.lookback
Default rolling window searched for the newest eligible datapoint. It must be at least the effective period, an exact period multiple, and no more than 1,440 buckets.
no
rule_defaults.query.publication_delay
Default collector wait after a bucket closes before it becomes eligible. This is a scheduling policy, not an AWS publication guarantee. Omission falls through to the profile value and then the built-in 10m fallback. Setting this option overrides profile-specific delays for every inheriting rule, including the stock S3 storage profile’s conservative 1d; AWS documents only that S3 storage metrics are reported once per day, so use a shorter default only after verifying each workload’s publication timing.
no
rules[].query
Optional query timing overrides for this rule. Each omitted field independently inherits rule_defaults.query, then the relevant profile or built-in fallback.
no
rules[].query.period
CloudWatch aggregation period for every series selected by this rule. Rate metrics are normalized using this effective period.
no
rules[].query.lookback
Rolling window searched for the newest complete finite datapoint. Successful queries may present the retained datapoint as current for up to this duration; longer lookbacks increase response work.
no
rules[].query.publication_delay
Collector wait after a bucket closes before querying it. This is a scheduling policy, not an AWS publication guarantee. Explicit 0s is allowed for metrics known to publish immediately.
no
Resource Filters
rule_defaults.filters.resource_tags
Job-wide list of exact, case-sensitive AWS resource tag predicates inherited by rules that omit rules[].filters.resource_tags. All keys must match; any listed value for one key may match. The Resource Groups Tagging API performs the focused lookup and requires tag:GetResources.
no
rule_defaults.filters.resource_tags[].key
Exact AWS resource tag key. A filter list supports at most 50 distinct keys.
yes
rule_defaults.filters.resource_tags[].values
One to 20 exact, case-sensitive accepted values for this key. Values for one key are ORed.
yes
rules[].filters.resource_tags
Per-rule replacement for rule_defaults.filters.resource_tags. Omit it to inherit the default, provide a non-empty list to replace the default, or set [] to disable tag filtering for this rule.
no
rules[].filters.resource_tags[].key
Exact AWS resource tag key. A filter list supports at most 50 distinct keys.
yes
rules[].filters.resource_tags[].values
One to 20 exact, case-sensitive accepted values for this key. Values for one key are ORed.
yes
Resource Labels
labels.resource_tags
Optional AWS resource tags copied to charts as non-identity labels. This is presentation only and does not select resources. Tag values may contain personal data, so expose only keys intended for Netdata. Requires tag:GetResources.
no
labels.resource_tags[].key
Exact, case-sensitive AWS resource tag key.
yes
labels.resource_tags[].label
Optional Netdata label key. When omitted, the AWS key is normalized (Name becomes name). Use an explicit label to avoid invalid names or collisions with identity labels such as region.
no
Limits
limits.max_instances
Maximum distinct final static or discovered CloudWatch instances that emit at least one selected series after filtering and exported-series overlap resolution. Metric/statistic fan-out is not counted. Overflow rejects the refreshed plan; collection never truncates to the first N instances.
1000
no
limits.max_discovery_groups
Maximum unique (target, region, namespace) discovery groups compiled for the job. Compatible rules and profiles share groups. The default is an accidental-expansion safeguard; raise it only for intentional work. Valid range 1–100. Split larger collection across jobs because one refresh can admit at most 100 groups that reach ListMetrics.
64
no
Discovery
discovery.refresh_every
How often (seconds) to re-discover metrics. Minimum 60.
300
no
discovery.recently_active_only
Use CloudWatch’s three-hour activity filter only when every selected series sharing a target, region, and namespace scan has publication_delay + lookback + period of 3 hours or less. Any longer-horizon participant keeps the shared scan unfiltered.
yes
no
Virtual Node
vnode
Associates this data collection job with a Virtual Node.
no
via UI
Configure the cloudwatch collector from the Netdata web interface:
Go to Nodes.
Select the node where you want the cloudwatch data-collection job to run and click the :gear: (Configure this node). That node will run the data collection.
The Collectors → Jobs view opens by default.
In the Search box, type cloudwatch (or scroll the list) to locate the cloudwatch collector.
Click the + next to the cloudwatch collector to add a new job.
Fill in the job fields, then click Test to verify the configuration and Submit to save.
Test runs the job with the provided settings and shows whether data can be collected.
If it fails, an error message appears with details (for example, connection refused, timeout, or command execution errors), so you can adjust and retest.
via File
The configuration file name for this integration is go.d/cloudwatch.conf.
The file format is YAML. Generally, the structure is:
Use one static/session credential source to assume roles for multiple monitored targets. Store credentials in supported secret providers, not in plaintext.
Use disjoint exact metric selections when one service needs different query timing. Earlier rules own only the metric/statistic series they select, so these two Lambda rules do not shadow each other.
Collect each available exact Billing grain independently. Billing metrics must be enabled first, are published only in us-east-1, and do not support resource-tag filters or resource-tag-derived labels. The stock profiles use a 10-minute period and 24-hour retrieval window; a rules[].query block would override only the fields it sets.
jobs:
- name: billing_estimated_chargescredentials:
- name: sdk_defaulttype: defaulttargets:
- name: billingcredentials: sdk_defaultrules:
- name: billing-grainstargets: [billing]profiles:
defaults: falseinclude:
- billing_total - billing_service - billing_linked_account - billing_linked_account_serviceregions: [us-east-1]filters:
# Billing is not an RGTA resource. This explicitly# disables any inherited resource-tag filter.resource_tags: []
AWS PrivateLink endpoints with split timing
Collect endpoint-level Average statistics every minute, collect six-hour processed-byte totals independently, and opt into the higher-cardinality endpoint-by-subnet view. Both endpoint grains support the same VPC endpoint resource-tag filters and labels; subnet charts inherit their parent endpoint’s tags.
Select defaults and explicitly add every disabled opt-in profile, including the PrivateLink endpoint-by-subnet view and four Billing grains. Their cardinality, prerequisites, and cost guidance still apply.
Charts are generated at runtime from the active service profiles. Each static or discovered AWS instance becomes a chart instance identified by its account_id, region, and the profile’s identifying dimensions (for example instance_id for EC2, or bucket_name and storage_type for S3); its contexts live under the cloudwatch. namespace. All CloudWatch metrics use the job’s configured vnode when present, otherwise the node running the collector. Individual AWS resources are distinguished by labels, not generated as separate Netdata nodes. Because CloudWatch publishes with a delay, allow a few minutes for the first data points.
Key terms:
Namespace – AWS’s grouping for a service’s metrics (e.g. AWS/EC2).
Dimension – a name/value pair that identifies a resource within a namespace (e.g. InstanceId).
Statistic – the CloudWatch aggregation applied per period (e.g. average, sum, maximum).
Profile – the Netdata YAML file that maps a namespace’s metrics to charts.
Partition – an isolated AWS region group (standard aws, GovCloud aws-us-gov, or China aws-cn); all regions selected for one target must share a partition, and an assumed-role ARN must match it.
The built-in profiles ship the following charts by default. Each service links to its profile – the authoritative definition of its exact metrics, statistics, dimensions, and charts:
control-plane health: API server request rate, errors, p99 latency, and in-flight requests; etcd database size; scheduler pending pods and scheduling attempts
Each profile also carries optional metrics that are commented out to keep cost and cardinality low; uncomment a metric and its matching chart, then restart the Netdata Agent (profiles are loaded once per go.d process and cached). Stock profiles are shipped at /usr/lib/netdata/conf.d/go.d/cloudwatch.profiles/default/. To customize a service, copy its profile into /etc/netdata/go.d/cloudwatch.profiles/ (keep the same filename) and edit it – a user profile fully replaces the stock one of the same name – then restart the Agent.
These disabled opt-in profiles are collected when a rule names them in profiles.include:
estimated charges by linked_account_id and service_name when available
The Billing profiles are exact grains rather than one wildcard: select only the views you need. All use EstimatedCharges, Maximum, USD, a 10-minute period, and a 24-hour retrieval window. AWS publishes the underlying estimate several times daily, so the collector normally re-emits the newest successful value between publications. Around the UTC month boundary, a prior-month value can remain visible until a successful query replaces or expires it, and transient AWS failures can retain it longer. region=us-east-1 identifies the CloudWatch publication/query location; the charge itself is worldwide. Billing dimensions are not AWS Resource Groups Tagging API resources, so resource-tag filters and resource-tag-derived labels do not apply. A valid job can produce no Billing chart when AWS has not published the selected grain.
PrivateLink endpoint metrics have two exact grains. The default privatelink_endpoint profile identifies one endpoint with endpoint_type, service_name, vpc_endpoint_id, and vpc_id. The opt-in privatelink_endpoint_subnet profile adds subnet_id; enable it deliberately because every endpoint can produce several subnet chart instances and seven additional metric/statistic queries per instance. Both profiles use a five-minute period, lookback, and publication delay by default. They share one AWS/PrivateLinkEndpoints discovery scan and one VPC endpoint Resource Groups Tagging API association, so endpoint resource-tag filters and labels also apply to every subnet child. The stock surface exports Average and per-second Sum views where both are useful; exact metric/statistic rules can assign different timing without shadowing siblings. These profiles do not query the separate AWS/PrivateLinkServices namespace.
At stock timing, each endpoint or opted-in subnet instance is queried once per five-minute window. Its seven selected series form five structural CloudWatch metrics for AWS billing grouping, or 1,440 metric requests per day before retries. A one-minute override runs its selected metrics five times as often; narrow profiles, metrics, regions, and resource tags when that freshness is not required.
These opt-in profiles include potentially high-cardinality data. S3 Request Metrics additionally require per-bucket request-metrics configuration in AWS and are billed at CloudWatch custom-metric rates; they collect nothing until enabled on the bucket. PrivateLink subnet cardinality grows with an endpoint’s attached subnets. The Billing service/account grains grow with the payer’s services and linked accounts; their cost guidance is described above.
SNS notifications failed to redrive to DLQ on ${label:topic_name}
Troubleshooting
Debug Mode
Important: Debug mode is not supported for data collection jobs created via the UI using the Dyncfg feature.
To troubleshoot issues with the cloudwatch collector, run the go.d.plugin with the debug option enabled. The output
should give you clues as to why the collector isn’t working.
Navigate to the plugins.d directory, usually at /usr/libexec/netdata/plugins.d/. If that’s not the case on
your system, open netdata.conf and look for the plugins setting under [directories].
cd /usr/libexec/netdata/plugins.d/
Switch to the netdata user.
sudo -u netdata -s
Run the go.d.plugin to debug the collector:
./go.d.plugin -d -m cloudwatch
To debug a specific job:
./go.d.plugin -d -m cloudwatch -j jobName
Getting Logs
If you’re encountering problems with the cloudwatch collector, follow these steps to retrieve logs and identify potential issues:
Run the command specific to your system (systemd, non-systemd, or Docker container).
Examine the output for any warnings or error messages that might indicate issues. These messages should provide clues about the root cause of the problem.
System with systemd
Use the following command to view logs generated since the last Netdata service restart:
journalctl _SYSTEMD_INVOCATION_ID="$(systemctl show --value --property=InvocationID netdata)" --namespace=netdata --grep cloudwatch
System without systemd
Locate the collector log file, typically at /var/log/netdata/collector.log, and use grep to filter for collector’s name:
grep cloudwatch /var/log/netdata/collector.log
Note: This method shows logs from all restarts. Focus on the latest entries for troubleshooting current issues.
Docker Container
If your Netdata runs in a Docker container named “netdata” (replace if different), use this command:
docker logs netdata 2>&1 | grep cloudwatch
No metrics are collected
Check the following:
Permissions – every target allows cloudwatch:GetMetricData; targets selecting any profile with dynamic dimensions also require cloudwatch:ListMetrics. All-constant profiles such as billing_total skip discovery and do not need ListMetrics. GetCallerIdentity needs no explicit grant. Targets with assume_role also require sts:AssumeRole on the source identity. Resource tag filtering or labels require tag:GetResources.
Rules – rules[].targets, rules[].profiles, optional rules[].metrics, and rules[].regions select the expected target, service, exact metric/statistic, and region. CloudFront publishes metrics only in us-east-1; its profile enforces this automatically.
Resource filters – a rule that omits filters.resource_tags inherits rule_defaults.filters.resource_tags. An explicitly included profile without a safe Resource Groups Tagging API association is rejected; use filters.resource_tags: [] for a deliberate unfiltered rule.
Resources are active – confirm in the AWS CloudWatch console that the resources are publishing metrics.
Collector logs – check for authentication or API errors:
Profile selection – omit rules[].profiles to select defaults, or ensure the service basename appears under rules[].profiles.include and is not excluded.
Metric selection – omit rules[].metrics to collect every metric exported by selected profiles. When configured, verify each profile group names a selected profile, every include[].name is an exact AWS MetricName, and every metric has effective statistics from either its replacement list or the group default.
Daily metrics – AWS documents that S3 storage metrics are reported once per day, but does not state a publication-within-one-day guarantee. The stock profile therefore uses a conservative 1d collector delay policy, and recently_active_only is automatically disabled for its long query horizon.
Resource activity – some metrics only appear when the resource is actively processing data (for example, EventBridge and Bedrock publish a metric only when its value is non-zero).
Auto Scaling group metrics – Auto Scaling group metrics (cloudwatch.auto_scaling.*) are not published until group-metrics collection is enabled on the group (aws autoscaling enable-metrics-collection --granularity 1Minute). Amazon EKS managed node groups have it enabled by default.
EKS control-plane metrics – EKS control-plane metrics (cloudwatch.eks.*) are published to the AWS/EKS namespace automatically, at no additional EKS charge, only for clusters running Kubernetes 1.28 or later; older clusters do not report them. These are distinct from Container Insights / the CloudWatch Observability add-on (agent-based, billed separately).
Charts have gaps or incomplete data
CloudWatch publishes metrics with a delay.
Keep rules[].query.period at or above the metric’s real publication cadence. A shorter override increases billed query frequency but cannot make AWS publish more often, so it can create empty windows.
Set rules[].query.publication_delay when a workload publishes completed buckets later than its profile or the built-in 10m fallback.
Check rule_defaults.query.publication_delay before overriding an individual rule. A job-wide value replaces profile-specific delays for inheriting rules, including the stock S3 storage profile’s conservative 1d, and a shorter value can query daily data before it is published.
Set rules[].query.lookback to search a wider rolling window for sparse datapoints. It must be an exact multiple of the effective period.
A successful query presents its newest eligible datapoint on every Netdata collection cycle, so an old CloudWatch value can appear current for up to lookback. During transient AWS failures, replay can continue longer until a successful query replaces or expires it.
Longer lookbacks increase response work and may disable recently_active_only for the shared discovery scan.
Transient query failures preserve the retained value and retry after one update_every; later delays double within the same eligible window up to the effective period. A newly eligible window resets the backoff.
Discovery group limit exceeded
A discovery group is one unique (target, region, namespace) combination. Compatible rules and profiles share the same group. limits.max_discovery_groups defaults to 64 to bound accidental ListMetrics expansion.
This can result from accidental target/region/profile expansion or from a legitimate large installation. Verify the derived scope first. For intentional scale, raise the safeguard up to 100. Beyond 100, split the configuration across multiple jobs: one bounded refresh admits at most 100 ListMetrics SDK operations, and every non-skipped group that resolves a client reaches its first admitted operation before continuations. Skipped groups and client-resolution failures consume no operation budget. Splitting preserves metric coverage while keeping each job’s discovery cost, memory, and completion time bounded.
Access denied or authentication errors
Verify the credential source referenced by the failing target is valid and not expired.
For a target with assume_role, confirm its source identity is allowed to assume the role and that the role trust policy permits it. If the trust policy requires an external ID, use the value supplied by the role owner.
For AWS GovCloud or China partitions, ensure each target’s selected rule regions match its role ARN partition.
The observability platform companies need to succeed
