# standard robot exclusion User-agent: * Disallow: /docs Disallow: /images Disallow: /VisualCalc Disallow: /javascript # allow LinkWalker - it checks all the links in the web site User-agent: LinkWalker Disallow: /images Disallow: /javascript