Back to Blog

research

Investigate VPC DNS Failures Before You Change Route 53

Use the aws-vpc-dns-investigation skill to debug VPC DNS issues and simulate Route 53 changes before they break production.

OpenAgentSkillPublished:

Where this fits

You are debugging why api.internal.example returns NXDOMAIN from one EC2 instance while a co-worker's instance resolves it fine. This skill lets an agent investigate VPC DNS from inside the subnet and test proposed Route 53 changes before you touch the control plane.

Why agents benefit

  • Runs dig, nslookup, and getent on target EC2 instances through SSM, so the agent sees resolution from the same VPC context as the app.
  • Simulates a change (e.g. enabling private DNS on a VPC endpoint, adding a Resolver rule) and predicts impact before it is applied.
  • Compares A and AAAA lookups to catch IPv4/IPv6 and dualstack differences.
  • Detects split-horizon problems by mapping which Resolver rule or hosted zone answers for a name.
  • Produces a repeatable RCA trail for incidents, saving time on postmortems.

Practical scenarios

NXDOMAIN after associating a private hosted zone

After adding a private hosted zone, an application starts failing with SERVFAIL. The agent can query the association from the subnet and determine if overlapping names or resolver rule priority caused the regression.

Public IP returned for an endpoint expected to be private

A service stops resolving to its VPC endpoint IP. The skill drives live checks and compares the current answer to the expected Route 53 endpoint behavior, helping isolate a new DNS Firewall rule or a removed Resolver rule.

Works from one instance, not another

When a hostname resolves only in certain subnets, the agent can run comparisons across instances and inspect subnet-level Resolver DNS64 / Route 53 Profile differences.

Add it to your agent workflow

Install via the skills CLI:

npx skills add aws/tools-for-devops-agent --skill aws-vpc-dns-investigation

Then ask something like: "Check whether enabling private DNS on my VPC endpoint will break db.internal.example. Simulate the change from the subnet where the app runs." The skill drives the aws-vpc-dns-diagnostics MCP server and reports what would change, not just what currently resolves.

Compare before adopting

Before adding this skill, compare it with general AWS troubleshooting skills. Look at maintenance freshness and whether it wraps the live MCP server or only suggests static checks. Consider alternatives like manual SSM commands or your own scripts, and validate that it fits how your agent handles incident RCA tasks. The GitHub repo, aws/tools-for-devops-agent, is the primary signal for activation.

Why it is worth tracking

With 42 stars and an AWS maintainer, this skill is early but purpose-built: the same team maintains the underlying MCP server. As DNS in VPCs becomes more complex with Route 53 Profiles, DNS Firewall, and hybrid forwarding, this skill's ability to simulate before applying makes it a strong candidate to watch. You can track it on OpenAgentSkill to see updates and evaluate whether it earns a permanent place in your toolkit.

Featured Skill

aws-vpc-dns-investigation

Skill by aws