LLMs.txt Generator API

Generate standardized llms.txt files to help Large Language Models effectively use your website content. This implementation follows the specification proposed by Jeremy Howard.

What is llms.txt?

The llms.txt file is a new standard that helps LLMs understand and use website content more effectively. It provides:

• A structured way to present project documentation
• Clear organization of API references and examples
• Distinction between core and optional documentation
• Machine-readable format that's also human-friendly

API Usage

Simply provide a URL to your project (GitHub repository or documentation site), and we'll generate an appropriate llms.txt file.

Endpoint

https://llms-txt-generator.estevecastells.workers.dev/

Simple Request Format

Just send a URL:

{ "url": "https://github.com/username/project" }

Example Response

# FastHTML > FastHTML is a python library which brings together Starlette, Uvicorn, HTMX, and fastcore's `FT` FastTags into a library for creating server-rendered hypermedia applications. Although parts of its API are inspired by FastAPI, it is *not* compatible with FastAPI syntax FastHTML is compatible with JS-native web components and any vanilla JS library, but not with React, Vue, or Svelte ## Docs - [FastHTML quick start](https://docs.fastht.ml/quickstart): A brief overview of many FastHTML features - [HTMX reference](https://raw.githubusercontent.com/htmx-org/htmx/master/www/content/reference.md): Brief description of all HTMX attributes ## API - [API List](https://docs.fastht.ml/apilist.txt): A succint list of all functions and methods ## Examples - [Todo list application](https://github.com/user/repo/examples/todo.py): Complete CRUD app example ## Optional - [Advanced Topics](https://docs.example.com/advanced): In-depth documentation

Implementation Examples

Python

import requests # Simple usage response = requests.post( "https://llms-txt-generator.estevecastells.workers.dev/", json={"url": "https://github.com/username/project"} ) # Print generated llms.txt print(response.json()["content"])

cURL

curl -X POST "https://llms-txt-generator.estevecastells.workers.dev/" \ -H "Content-Type: application/json" \ -d '{ "url": "https://github.com/username/project" }'

Supported Sources

• GitHub Repositories: Automatically extracts information from:
  • Repository description and README
  • Documentation in docs/ directory
  • Example code in examples/ directory
  • API documentation
• Documentation Sites: Supports:
  • GitHub Pages
  • Custom documentation sites
  • Standard documentation paths

Best Practices

• For GitHub projects, ensure your README is up to date and descriptive
• Use standard directory names like 'docs', 'examples', 'api' for better detection
• Include meta descriptions in your documentation pages
• Organize documentation into clear sections
• Use the Optional section for advanced or supplementary content

Format Specification

The generated llms.txt follows this structure:

• Title (H1): Project name
• Description (Blockquote): Brief project overview
• Details: Important points about usage/compatibility
• Sections (H2):
  • Docs: Main documentation
  • API: API references
  • Examples: Code examples and demos
  • Optional: Additional resources

Error Handling

The API returns clear error messages for common issues:

{ "error": "Failed to process URL", "details": "Specific error message" }

For questions, issues, or feature requests, please use the contact information provided on the website.