LLMs-code.txt (#2819)

* Add new docContet.js util

* Add new openLLM action

* Swap generic icon for ChatGPT with the official OpenAI logo from PhosphorIcons library

* Swap sparkle icon for Claude with a chat bubble icon, since sparkle is used for Kapa

* Add prompt localization for the top 10 most used languages

* Fix Claude incorrectly building translated prompt

* Better fix Claude handling of locales by adding dual English + localized prompt

* Ensure only Claude shows dual prompts

* Hide descriptions for Open with… buttons

* Update icon for Claude

* Fix icon position when no description provided

* Try to accomodate for both description and no desciption scenarii for icons placement

* Revert "Try to accomodate for both description and no desciption scenarii for icons placement"

This reverts commit cdb8d76.

* Simplify icons alignment fix

* Try to fix Claude's issue

* Fix query parameter for Claude

* Remove dual prompt in Claude now that we fixed the query param

* Refactor translation system and add many more locales

* Add aiPromptTemplates.js

* Add script for translated prompt validation

* Added many more languages

* Include Tldr components content in llms.txt

* WIP: Add draft of generate-llms-code.js to work on 3 sample files

* Improve language detection

* Improve formatting and language detection

* Improve generation

* Enhance file path and title logic

* Improve fenced code blocks detection and generation

* Handle dependencies "lazy loading"

* Improve metadata parsing

* Print out llms-code to stdout

* Improve file path and language detection further

* Handle more languages in a heuristic way

* Add normalizeOutputPath to better handle carelss mistakes and add support for powershell blocks

* Add support for anchors in sources

* Handle custom headings

* Add anchors tag to generation script in package.json

* Improve SQL block language inference

* Append empty strings to ensure we don't miss language blocks

* Better handle Docusaurus tabs detection

* Add additional scripts to handle file existence and project root

* Add validator script

* Add script to both generate and validate

* Add llms-code validator and all-docs discovery

Adds include/exclude filters and wires validation into dev/build.

* Add optional line numbers to llms-code output

Emits "Lines: start-end" when --line-numbers is used.

* Add section-level Description and Source lines

* Emit Language and File path per variant; remove extra example headers

* Normalize js/ts vs JavaScript/TypeScript in validator and don't treat missing local files as error unless check-files is enabled

* Relax default validator: skip file existence checks by default; add validate:llms-code:strict for CI

* Run generator with --all and propagate flags from CLI to generator config

* Recognize GraphQL/HTML/DOTENV/TEXT and alias sh→bash; normalize js/ts aliases in validator

* Avoid EISDIR during anchor verification by only accepting files (not directories) when resolving doc paths

* Downgrade missing anchor from error to warning during anchor verification

* Guard against incomplete fences: skip empty variants in generator and continue after unclosed fence in validator

* Relax severity: treat missing/unclosed code fences and empty sections as warnings to avoid false-negative failures on prose-only sections

* Strengthen section Description fallback and skip sections with no valid variants to avoid missing Description errors

* Limit --all discovery to cms/ and cloud/; exclude snippets and other directories

* Require Description only for sections with variants; suppress fence-related warnings for prose-only sections

* Do not require section Description; suppress anchor-not-found warnings for cleaner zero-warning runs

* Fix validator crash when section Description is omitted by guarding desc lookups

* Accept both '(Source: …)' and 'Source: …' formats in validator to handle section headers and example variants

* Make section Source optional: accept none, '(Source: …)' or 'Source: …)' and skip URL checks when absent

* Allow fence-first variants (no 'Language:' line) and soften non-absolute Source to warning when present

* Accept missing or legacy file path line per variant (treat as N/A) and only check existence when provided

* Add --verbose flag; collapse per-file 'no snippets' notices into a single summary line with count

* Add --log-file option to write skipped doc IDs; keep concise summary unless --verbose is used

* Auto-create skip log in verbose mode (static/llms-code-skip.log) and mention path; ignore log file in git

* Add llms:generate-verbose script to run generator with --verbose and point to skip log

* Add llms:generate-and-validate:verbose script (verbose generation + validation)

* Add concise docs for llms-code generator/validator, scripts, and flags

* Group llms scripts under scripts/llms and update npm scripts to use wrappers

* Use yarn script for LLMs generation in deploy workflow; target explicit files when checking and committing

* Generate & validate llms-code.txt in deploy workflow; include in change detection and commit alongside llms.txt and llms-full.txt

* Add 'View LLMs-code.txt' button in AI toolbar (navigate to /llms-code.txt)

* Restore proper .gitignore content

* Move llms-code button down

* llms-code: ensure generation writes to static by fixing docs dir discovery (auto-fallback to docusaurus/docs) and wire into dev/build scripts

* Delete docusaurus/docs/contributing/ai-toolbar-translations.md

* Delete docusaurus/docs/ai-toolbar-translations.md

* Delete docusaurus/scripts/validate-prompts.js

* Rename llms-code.md to README-llms-code.md

---------

Co-authored-by: Portal Code Bot <[email protected]>

Author: pwizla

.github/workflows/deploy-production.yml

@@ -30,12 +30,18 @@ jobs:
 
       - name: 🤖 Generate LLMs files
         working-directory: ./docusaurus
-        run: node scripts/generate-llms.js
+        run: npm run generate-llms
+
+      - name: 🤖 Generate LLMs code file
+        working-directory: ./docusaurus
+        run: |
+          node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt
+          node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root ..
       
       - name: 🔍 Check for changes
         id: check-changes
         run: |
-          if git diff --quiet HEAD -- docusaurus/static/llms*.txt; then
+          if git diff --quiet HEAD -- docusaurus/static/llms.txt docusaurus/static/llms-full.txt docusaurus/static/llms-code.txt; then
             echo "changed=false" >> $GITHUB_OUTPUT
             echo "🔄 No changes in LLMs files"
           else
@@ -48,7 +54,7 @@ jobs:
         run: |
           git config --local user.email "[email protected]"
           git config --local user.name "GitHub Actions"
-          git add docusaurus/static/llms*.txt
+          git add docusaurus/static/llms.txt docusaurus/static/llms-full.txt docusaurus/static/llms-code.txt
           git commit -m "🤖 Update LLMs files [skip ci]"
           git push
 

docusaurus/package.json

@@ -4,8 +4,8 @@
   "private": true,
   "scripts": {
     "docusaurus": "docusaurus",
-    "dev": "docusaurus start --port 8080 --no-open",
-    "build": "docusaurus build",
+    "dev": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root .. && docusaurus start --port 8080 --no-open",
+    "build": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root .. && docusaurus build",
     "swizzle": "docusaurus swizzle",
     "deploy": "docusaurus deploy",
     "clear": "docusaurus clear",
@@ -14,9 +14,14 @@
     "write-heading-ids": "docusaurus write-heading-ids",
     "release-notes": "bash ./scripts/release-notes-script.sh",
     "redirections-analysis": "node ./scripts/redirection-analysis/redirect-analyzer.js",
-    "generate-llms": "node scripts/generate-llms.js",
-    "dev:with-llms": "yarn generate-llms && docusaurus start --port 8080 --no-open",
-    "build:with-llms": "yarn generate-llms && docusaurus build",
+    "generate-llms": "node scripts/llms/generate-llms.js",
+    "dev:with-llms": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root .. && docusaurus start --port 8080 --no-open",
+    "build:with-llms": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root .. && docusaurus build",
+    "llms:generate-and-validate": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --output static/llms-code.txt && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root ..",
+    "llms:generate-verbose": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --verbose --output static/llms-code.txt && echo 'Skip log (if any): static/llms-code-skip.log'",
+    "llms:generate-and-validate:verbose": "yarn generate-llms && node scripts/llms/generate-llms-code.js --anchors --all --verbose --output static/llms-code.txt && echo 'Skip log (if any): static/llms-code-skip.log' && node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root ..",
+    "validate:llms-code": "node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --verify-anchors --project-root ..",
+    "validate:llms-code:strict": "node scripts/llms/validate-llms-code.js --path static/llms-code.txt --strict --check-files --verify-anchors --project-root ..",
     "meilisearch:update-order": "node -r dotenv/config scripts/meilisearch/add-category-order.js"
   },
   "dependencies": {

docusaurus/scripts/README-llms-code.md

@@ -0,0 +1,79 @@
+# llms-code: generator and validator
+
+This tooling extracts code examples from docs and emits a single consumable file for LLMs, plus an optional validation pass to catch structural issues early.
+
+## What it generates
+
+- `static/llms-code.txt` with blocks grouped by page and section:
+  - `## Section`
+  - `Description: ...` (optional)
+  - `(Source: https://docs.strapi.io/...#anchor)` when `--anchors` is set
+  - For each variant: `Language: ...`, `File path: ...` (or `N/A`), fenced code, `---` divider between variants
+- In verbose runs a skip log is written to `static/llms-code-skip.log` listing pages with no code snippets.
+
+## npm/yarn scripts
+
+Run from `docusaurus/`:
+
+- `yarn llms:generate-and-validate`
+  - Generate for all docs and validate (quiet output; no file existence checks)
+
+- `yarn llms:generate-verbose`
+  - Generate with `--verbose`; prints each skipped page and writes `static/llms-code-skip.log`
+
+- `yarn llms:generate-and-validate:verbose`
+  - Verbose generate (writes skip log) then validate (quiet)
+
+- `yarn validate:llms-code`
+  - Validate an existing `static/llms-code.txt` (quiet; no file existence checks)
+
+- `yarn validate:llms-code:strict`
+  - Validate with file existence checks (use only if paths point to a real project), plus anchor verification
+
+## Generator flags (`scripts/generate-llms-code.js`)
+
+- `--all` Scan all docs (restricted to `cms/` and `cloud/` trees)
+- `--include a,b` / `--exclude x,y` Filter discovered doc IDs by substring
+- `--anchors` Include section anchors in Source lines
+- `--line-numbers` Emit `Lines: start-end` for each variant
+- `--verbose` Print per-file skip messages; auto-writes `static/llms-code-skip.log`
+- `--log-file path` Custom path for the skip log
+- `--output path` Destination for generated text (use `-` for stdout)
+
+Notes:
+- Discovery intentionally excludes `snippets/` and other non-doc trees.
+- When file path cannot be inferred it is emitted as `N/A`.
+
+## Validator flags (`scripts/validate-llms-code.js`)
+
+- `--path path` Input file (use `/dev/stdin` to validate from a pipe)
+- `--strict` Exit with non-zero on any errors (warnings do not fail)
+- `--verify-anchors` Check that section anchors exist in the source doc
+- `--check-files` Check referenced files exist (use with `--project-root ..` when appropriate)
+- `--project-root dir` Base path for file checks and anchor lookups
+- `--report json|text` Output diagnostics as JSON or text (default)
+
+Heuristics and niceties:
+- Language aliases normalized (e.g., `js` ≡ `JavaScript`, `ts` ≡ `TypeScript`, `sh` treated as `Bash`, `graphql`, `html`, `dotenv`, `text` recognized)
+- Fence-first blocks (without a `Language:` line) are accepted by inferring language from the fence
+- Section Description and Source are optional and do not fail validation
+- File path line may be `File path:` or legacy `File:`; missing is treated as `N/A`
+
+## Examples
+
+- Generate + validate (quiet):
+```
+yarn llms:generate-and-validate
+```
+
+- Verbose generate + validate, with skip log:
+```
+yarn llms:generate-and-validate:verbose
+```
+
+- Validate a streamed output without writing a file:
+```
+node scripts/generate-llms-code.js --anchors --all --output - \
+  | node scripts/validate-llms-code.js --path /dev/stdin --strict --verify-anchors --project-root ..
+```
+