Kopier kodene generatoren returerer og lim dem inn i en tom fil som du kaller robots.txt. Du kan her benytte Notepad. Åpn dette standard programmet som følger med alle Windows versjoner og lim inn kodene i den blanke siden som kommer opp.

Lagr denne filen som robots.txt, og last så denne filen opp til ditt nettsted.

Filen skal ligge i root-katalogen sammen med din index/default fil.

Andre, litt mer avanserte generatorer er:

En meget avansert robots.txt generator finner du under.

PS!
Husk å sjekk at din robots.txt fil er korrekt til slutt, så du er sikker på at den er skrevet korrekt.

Robots.txt generator - avangsert versjon

Robots.txt Generator Tool

Default Robots.txt file setting

Default - All Robots are:

Crawl-Delay:

Specific Search Robots

Search Engine	Choose Setting	Bot-Name
Google		googlebot
MSN Search		msnbot
Yahoo		yahoo-slurp
Alexa/Wayback		ia_archiver
Ask/Teoma		teoma
DMOZ		robozilla
GigaBlast		gigabot
Scrub The Web		scrubby

Specific Special Bots:

Search Engine	Choose Setting	Bot-Name
Google Image		googlebot-image
Yahoo MM		yahoo-mmcrawler
MSN Pic-Search		psbot
Yahoo Blogs		yahoo-blogs/v3.9

Extra Exclusion features:

Search Engine	Choose Setting	Bot-Name
Google		googlebot
MSN		msnbot
Yahoo		yahoo-slurp

Disallow Bad Bots:
Select this option to block the most common "Bad-Bots," also known for spam and email harvesting from accessing your domain.
Disallow Bad-Bots:	User-agent: Black Hole User-agent: Titan User-agent: WebStripper User-agent: NetMechanic User-agent: CherryPicker User-agent: EmailCollector User-agent: EmailSiphon User-agent: WebBandit User-agent: EmailWolf User-agent: ExtractorPro User-agent: CopyRightCheck User-agent: Crescent User-agent: Wget User-agent: SiteSnagger User-agent: ProWebWalker User-agent: CheeseBot User-agent: Teleport User-agent: TeleportPro User-agent: MIIxpc User-agent: Telesoft User-agent: Website Quester User-agent: WebZip User-agent: moget/2.1 User-agent: WebZip/4.0 User-agent: WebSauger User-agent: WebCopier User-agent: NetAnts User-agent: Mister PiX User-agent: WebAuto User-agent: TheNomad User-agent: WWW-Collector-E User-agent: RMA User-agent: libWeb/clsHTTP User-agent: asterias User-agent: httplib User-agent: turingos User-agent: spanner User-agent: InfoNaviRobot User-agent: Harvest/1.5 User-agent: Bullseye/1.0 User-agent: Mozilla/4.0 (compatible; BullsEye; Windows 95) User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0 User-agent: CherryPickerSE/1.0 User-agent: CherryPicker /1.0 User-agent: WebBandit/3.50 User-agent: NICErsPRO User-agent: Microsoft URL Control - 5.01.4511 User-agent: DittoSpyder User-agent: Foobot User-agent: WebmasterWorldForumBot User-agent: SpankBot User-agent: BotALot User-agent: lwp-trivial/1.34 User-agent: lwp-trivial User-agent: Wget/1.6 User-agent: BunnySlippers User-agent: Microsoft URL Control - 6.00.8169 User-agent: URLy Warning User-agent: Wget/1.5.3 User-agent: LinkWalker User-agent: cosmos User-agent: moget User-agent: hloader User-agent: humanlinks User-agent: LinkextractorPro User-agent: Offline Explorer User-agent: Mata Hari User-agent: LexiBot User-agent: Web Image Collector User-agent: The Intraformant User-agent: True_Robot/1.0 User-agent: True_Robot User-agent: BlowFish/1.0 User-agent: JennyBot User-agent: MIIxpc/4.2 User-agent: BuiltBotTough User-agent: ProPowerBot/2.14 User-agent: BackDoorBot/1.0 User-agent: toCrawl/UrlDispatcher User-agent: WebEnhancer User-agent: TightTwatBot User-agent: suzuran User-agent: VCI WebViewer VCI WebViewer Win32 User-agent: VCI User-agent: Szukacz/1.4 User-agent: QueryN Metasearch User-agent: Openfind data gathere User-agent: Openfind User-agent: Xenu's Link Sleuth 1.1c User-agent: Xenu's User-agent: Zeus User-agent: RepoMonkey Bait & Tackle/v1.01 User-agent: RepoMonkey User-agent: Zeus 32297 Webster Pro V2.9 Win32 User-agent: Webster Pro User-agent: EroCrawler User-agent: LinkScan/8.1a Unix User-agent: Keyword Density/0.9 User-agent: Kenjin Spider User-agent: Cegbfeieh

Restricted Directories:

The path is relative to root and must contain a trailing "/"

Exclude Directory: "/"
Exclude Directory: "/"
Exclude Directory: "/"
Exclude Directory: "/"
Exclude Directory: "/"

Create The Robots.txt

Now, copy and paste this text above into a blank text file called "robots.txt" and put it in the root directory of your website. Make sure it's permissions are set so that visitors and search engines can read it.

Example: http://www.yoursite.com/robots.txt

Sjekk din robots.txt fil

Sjekk din robots.txt fil etter at du har limt inn kodene i verktøyet over for å sjekke at robots.txt filen ikke inneholder syntaks feil eller andre tekniske feil.

Skriv inn hele URLen til ditt nettsted, f.eks. http://www.dittdomene.no/: