Skip to content

Services
Projects
Portfolio
Tools
Blog
Contact

Loading

Web, mobile, SharePoint, AI and automation delivered by one senior engineer in Canberra.

Available for contract work

Pages

Services
Projects
Portfolio
Tech stack
Tools
Blog
Contact

Contact

jayden@mdt.id.au +61 431 149 567 LinkedIn GitHub Google Business ProfileCanberra, ACT, Australia7:00 am – 7:00 pm (Mon–Sun)

© 2026 MDT Software Development · Jayden MarquardtPrivacy

Home
Tools
Robots, Sitemap & Meta Fetcher

Robots, Sitemap & Meta Fetcher

Fetch robots.txt, llms.txt, sitemaps, feeds and page metadata, and see which AI crawlers are blocked.

Loading tool

More site & seo scanners

Site Intelligence Report
One scan, a hundred-odd checks: what Google sees, what AI assistants see, speed, security, brand, and a map of how every page links together, scored, sourced and exportable.
Open tool
Seo & OG Preview Tool
Preview how your website will look in search engines and social media
Open tool
Broken Link Checker
Crawl your site for broken links and report them
Open tool

Let’s build something amazing.

Have a project idea or need expert guidance? One message, one engineer, no account manager.

jayden@mdt.id.au
·
+61 431 149 567
·
7:00 am – 7:00 pm (Mon–Sun)

Site address*

Only public files a crawler could already read are fetched.

Pulls robots.txt, llms.txt, sitemaps, feeds, the manifest, security.txt and the home page meta in one pass.