Webflow SEO: robots.txt, llms.txt, schema and the settings that matter

Webflow SEO is set up in Site settings and in each page's settings: titles and descriptions, dynamic on CMS templates; robots.txt; an automatic sitemap; a global canonical; 301 redirects and llms.txt. Structured data goes in as custom code. Be careful with the AI bots toggle: it includes crawlers that cite you.

To check it on your own site: SEO audit

Webflow is a visual builder with its own hosting: you design in the Designer and Webflow publishes the pages on its servers. You get more control than on most builders, but the server is still theirs. The menu paths in this guide are the ones in Webflow's official help centre.

Titles and descriptions: static pages and CMS templates

On a static page, you write the title and description in the Pages panel, Page settings, "SEO settings". Webflow has no site-wide title: every page carries its own1.

On CMS templates (collection pages, such as each blog post), you define a pattern every item uses. In the template's "SEO settings", "Add field" inserts the value of a collection field, for instance the post name as the SEO title1.

A practical pattern is to add two plain text fields to the collection, "SEO title" and "SEO description", and connect them there. Each post then gets its own title and description without touching the template.

robots.txt and bot traffic control

You write robots.txt in Site settings, SEO, "Indexing". Webflow adds the sitemap line itself, so do not repeat it or you will end up with two23.

The same section holds the Traffic control toggles, which generate rules for groups of known bots. There are two groups, and both are allowed by default4.

• Search engine crawlers: Googlebot, Bingbot, YandexBot, DuckDuckBot and OAI-SearchBot, among others.
• AI bots: GPTBot, ClaudeBot, CCBot, meta-externalagent and Bytespider, but also PerplexityBot, Claude-User and ChatGPT-User.

The second group mixes two things. GPTBot and ClaudeBot are training crawlers: blocking them does not take you out of answers56. PerplexityBot is the crawler that links sites in Perplexity's answers7, and Claude-User fetches pages when someone asks Claude a question; Anthropic warns that blocking it may reduce your visibility6. Switch off the "AI bots" group and you leave those answers too. Claude-SearchBot is in neither group.

If you only want to block training, leave both toggles on and write your rules in the robots.txt field:

# Everyone else, including Googlebot and AI search crawlers
User-agent: *
Allow: /

# Model training: blocking it does not remove you from answers
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: Google-Extended
User-agent: Applebot-Extended
User-agent: CCBot
User-agent: meta-externalagent
Disallow: /

We ran it through Python's robots.txt parser: Googlebot, OAI-SearchBot, Claude-SearchBot, Claude-User and PerplexityBot come out allowed and the six training crawlers blocked. Google-Extended is a token, not a crawler: it opts you out of Gemini training and grounding in the Gemini app without affecting Google Search8.

Webflow also offers a Content-Signal header (ai-train=no, search=yes, ai-input=no) in the same section. Webflow itself notes it is based on a proposed extension to RFC 9309 that is not yet a standard2, and none of the OpenAI, Anthropic or Perplexity crawler pages cited here mention it. Add it if you like, but robots.txt is still what those crawlers say they respect.

Sitemap, noindex and canonical

• Sitemap. Site settings, SEO, "Sitemap": switch on "Auto-generate sitemap". Webflow rebuilds it on every publish and adds <lastmod> to each URL3.
• noindex on a page. Page settings, "SEO settings", "Sitemap indexing" switched off: it adds a robots noindex meta tag and removes the page from the sitemap4. A Disallow in robots.txt does not remove it from the sitemap, and Google cannot read the noindex on a page it is not allowed to crawl9.
• noindex on a CMS item. The same toggle in the item details. Paid site plans only4.
• webflow.io subdomain. Switch off "Staging indexing" so Google does not index your staging copy4.
• Global canonical. In "Global canonical tag URL", enter the base URL of your default domain, with https://, with www if your domain uses it and without a trailing slash, or the canonicals will come out with a double slash10.

301 redirects

They live in Site settings, Publishing, "301 redirects": old path, new path, "Add redirect path" and publish11. Paths are relative to the root domain. To redirect a page that still exists, first change its slug, delete it or save it as a draft11.

Structured data from CMS fields

Webflow does not generate JSON-LD for you. There are two places to add it:

1. A single page: Page settings, "Custom code", "Inside head tag". This needs a Site plan or a paid Workspace12.
2. A CMS template: a Code Embed element on the collection page. When you edit its code, the purple dot on each line opens a menu that inserts a field value from the item, which Webflow substitutes on publish13.

An example for a blog template. Replace each FIELD_… with the matching field using that menu:

<script type="application/ld+json">
{
  "@context": "https://schema.org",
  "@type": "BlogPosting",
  "headline": "FIELD_TITLE",
  "datePublished": "FIELD_DATE",
  "image": "FIELD_IMAGE",
  "author": { "@type": "Person", "name": "FIELD_AUTHOR" }
}
</script>

Two checks before you call it done. Publish an item whose title contains quotation marks and run it through the Rich Results Test: if the JSON breaks, remove the quotes from that field or use a different one. And make sure the date comes out in ISO 8601, which is what Google expects. Google accepts JSON-LD in the <head> or the <body>14, so a Code Embed in the page body is fine.

llms.txt on Webflow

Webflow lets you upload an llms.txt: Site settings, SEO, "LLMs.txt", "Upload file", a UTF-8 file under 100 KB. After publishing it sits at https://yourdomain.com/llms.txt; it is not published to the staging subdomain15.

Webflow's own help describes it as an experimental convention, not widely adopted and with unproven impact15. Google says no AI text files are needed to appear in AI Overviews or AI Mode16. It is cheap to do; do not expect it to deliver what a correct robots.txt and content in the HTML do not.

Publishing options: HSTS, minification and headers

Site settings, Publishing, "Advanced publishing options" gives you these settings17.

• SSL is on and cannot be switched off.
• Minify HTML, CSS, JS reduces code weight; each type is toggled separately.
• Asynchronously load JavaScript (paid plan) loads scripts in parallel; it can break custom code that depends on load order.
• Per page CSS splits CSS per page; check custom styles after enabling it.
• Use secure frame headers stops other sites from putting your pages in an iframe; it cannot be switched off with Ecommerce or User Accounts.

HSTS is on automatically for every non-Enterprise site and cannot be removed18. What you choose are two extras: extending it to subdomains and preload. With preload, browsers can end up forcing HTTPS on the whole domain and its subdomains, and getting off that list is slow. Webflow warns that, combined with subdomains, it can make your site unreachable if any subdomain uses HTTP18. Do not switch it on until you have checked every subdomain.

Custom security headers such as Content-Security-Policy are Enterprise only18. If you have them, do not add X-XSS-Protection: MDN marks it as deprecated and recommends Content-Security-Policy instead19.

How to check it

• robots.txt: open https://yourdomain.com/robots.txt and confirm PerplexityBot and Claude-User do not have Disallow: /.
• Response per bot: curl -I -A "PerplexityBot/1.0" https://yourdomain.com/ should return a 200.
• Canonical: "View page source" on a subpage and look for rel="canonical", with no double slash.
• CMS JSON-LD: Google's Rich Results Test on two or three different items.
• Headers: Mozilla's HTTP Observatory or curl -I.
• Speed: PageSpeed Insights, against Google's thresholds: LCP of 2.5 s or less, INP of 200 ms or less and CLS of 0.1 or less, at the 75th percentile20.

What SmoothSeen does with this

SmoothSeen analyses a page on your Webflow site. Under AI visibility it reads robots.txt with search crawlers kept apart from training crawlers, requests the page with each crawler's user-agent to catch a 403, measures how much text arrives in the HTML and checks for llms.txt. Under Web visibility it checks canonical, sitemap, structured data, security headers through Mozilla's HTTP Observatory and speed with PageSpeed Insights.

What to do this week

Go to Site settings, SEO, "Indexing" and look at the Traffic control toggles. If you switched off "AI bots" to stop training, switch it back on and use the robots.txt above. To see which AI crawlers can read your page, analyse it with SmoothSeen.

If you are comparing platforms, there are guides for Wix and Squarespace.

Frequently asked questions

Is Webflow good for SEO?

Yes, for what is in your hands: per-page titles and descriptions, dynamic ones in the CMS, an editable robots.txt, an automatic sitemap, a global canonical, 301 redirects and custom code for structured data. Content also arrives in the HTML: on 7 October 2026 Webflow's own blog, built with Webflow, served more than 2,000 words without running JavaScript. What you do not control is the server, except headers on Enterprise.

Will switching off AI bots in Webflow remove me from ChatGPT?

Not from ChatGPT search, because OAI-SearchBot sits in the search engine group, not the AI group. But the "AI bots" group includes PerplexityBot and Claude-User, so switching it off can take you out of Perplexity's and Claude's answers. To block training only, write rules for GPTBot, ClaudeBot and the rest in robots.txt.

How do I add schema to CMS pages in Webflow?

With a Code Embed element inside the collection template. You write the JSON-LD and, for each value that changes, insert the CMS field through the code editor's connect menu. Webflow replaces it with the real value on publish. Then test several items with the Rich Results Test.

Is uploading an llms.txt to Webflow worth it?

There is no evidence that it helps. Webflow offers it and warns that it is experimental with uncertain impact; Google says it does not need one, and no AI search engine has documented reading it. If you upload one, make it a plain summary of what you do with links to your main pages, not pasted marketing copy.