Programmatic Technical SEO: Internal Links & Sitemaps at Scale

When a publication grows past a few thousand URLs, nobody can hold the whole site graph in their head. New articles launch as orphans with no inbound links, and older high-authority pages point at redirects or retired URLs.

Treat internal linking as an indexing problem

Instead of asking editors to remember every related article, make linking a pipeline:

• On publish or update: index the article's topics, entities, and canonical slug, then suggest contextual links in both directions.
• On retirement (404 or 410): prune dead outbound anchors and forward link equity to the best replacement.
• At the transform layer: normalize CMS body links from root-relative paths to absolute canonical URLs.

Every automated change should be reversible. A rollback step is what makes it safe to run this on a live editorial catalog.

Sitemap hygiene

One of the fastest ways to waste crawl budget is a sitemap full of redirected or 410 Gone URLs. Only emit live, canonical, indexable records, and tie lastModified to real content changes.

// src/app/sitemap.ts
export default async function sitemap(): Promise<MetadataRoute.Sitemap> {
  const articles = await getIndexableArticles(); // excludes 404, 410, and redirects
  return articles.map((post) => ({
    url: `https://www.example.com/articles/${post.slug}`,
    lastModified: new Date(post.updatedAt ?? post.publishedAt),
  }));
}

Entity clarity for search and AI answer engines

Search engines and LLM-backed answer engines rely on unambiguous entities. Declare the right @type on each page (Person, WebSite, Article, BreadcrumbList), link profiles through sameAs, and keep self-referencing canonicals consistent with your sitemap.