Basics

Create a project

Pick what a project crawls: the whole site, one language version or one directory.

1. Enter the address

Go to Projects and choose New project. Enter the website's address, either a bare domain like example.com or a full address like https://example.com/en/, and choose Check site.

LinkTaste reads the homepage once to find the site's language versions and sitemaps. Nothing is crawled at this point.

2. Choose what to crawl

The next step shows what the check found: the site's title, its language and how many sitemaps it lists. Below that you choose the project's scope, the part of the site the crawl stays inside:

  • The whole site. Every page on the host.
  • A language version. When the site declares language versions (with hreflang) that live in directories such as /de/ and /en/, each one is offered on its own. Choosing one crawls only that directory. For the root version, the other language directories are skipped, and the row says which.
  • The directory you entered. If your address included a path, that directory is offered as a scope.
  • Another directory. Type any directory, for example /blog/, to analyse one section only.

The option marked Recommended is the one that matches what you entered. A multilingual site is best analysed one language at a time: each language has its own navigation and links mostly within itself, so mixing them blurs every figure.

Language versions that live on separate hosts (for example example.de and example.fr) are listed under the options. Create a separate project for each of them.

3. Name it and create it

The project name follows the scope you picked; change it if you like, and choose Create project.

4. Add the link snippet

The last step shows the project's link snippet: one script tag that writes the links you approve into your pages. Add it to every page, before </body>, and choose Test the connection. Or choose Skip for now: the tag stays under Link snippet on the project page. See Your links & the snippet.

Either way you land on the project page, ready to start the first crawl.

When the check reports a problem

  1. We couldn't read the homepage. The site did not answer, or answered with an error. You can still create the project; the crawl will report if the site keeps blocking it.
  2. robots.txt blocks our crawler here. The site's robots.txt disallows LinkTasteBot at this address. LinkTaste respects it, so a crawl would stop at the start page. Allow LinkTasteBot in robots.txt first; see LinkTasteBot for the rules it follows.

The project page

A project's page shows its scope (start URL, directory, what it skips, language), the latest crawl with its headline figures and what changed since the crawl before, and the history of every crawl. Its tabs hold the crawl Schedule (off, weekly or monthly) and the Link snippet with its connection status. Delete at the bottom removes the project with every crawl snapshot; it cannot be undone.

A project's scope is fixed once it is created. To crawl a different section of the same site, create another project for it.