3.7.8. Noindex and nofollow tags
Search engines crawl through the world wide web and create an index that is then also searchable by a user. Meta robot tags are so-called control commands for search engine robots, i.e. Google, Bing and Yahoo, which control the indexing. Using these meta tags you can advise if Google should include your pages in the Google search index or not. Each of your CMS page, category and product can be controlled separately or in a bulk adjustment within the Magento 2 admin panel.
Access to your CMS page which you wish to change the meta robots tags. To do so navigate to CONTENT / ELEMENTS / PAGES, edit your desired page and now expand SEARCH ENGINE OPTIMIZATION tab.
Here you will find a drop-down menu with following options:
INDEX, FOLLOW
o INDEX: Allows search engines that information about this page may be included in the
search index
o FOLLOW: Allows search engine robots to follow all links on this page
INDEX, FOLLOW
o INDEX: Allows search engines that information about this page may be included in the
search index
o NOFOLLOW: The robots of search engines shall not follow any links of this page
INDEX, FOLLOW
o NOINDEX: This page is not allowed to be included in the search index
o FOLLOW: Allows search engine robots to follow all links on this page
INDEX, FOLLOW
o NOINDEX: This page is not allowed to be included in the search index
o NOFOLLOW: The robots of search engines shall not follow any links of this page
If you wish to determine more than one URL you can also provide them in the Robots Meta Tags configurations. In order to do so navigate to STORES / SETTINGS / CONFIGURATIONS and then to MAGESUITE.
Here you can provide your input using the relative URL path with parameters, e.g. “en_en/women.html;INDEX_FOLLOW” or “contact-us.html;INDEX_NOFOLLOW”.
When your changes are done click “SAVE CONFIG” in the top-section on the right-hand side.
In order to control the tags to be masked on category level access CATALOG / INVENTORY / CATEGORIES and select a category from the category tree. Open the section SEARCH ENGINE OPTIMIZATION where you can find the area META ROBOTS. Here you can select the tags to be used.
When the changes are done click “SAVE” in the upper section on the right-hand side.
In order to control the tags to be masked on product level access CATALOG / INVENTORY / PRODUCTS and select a product from the product grid. Open the section SEARCH ENGINE OPTIMIZATION where you can find the area META ROBOTS. Here you can select the tags to be used.
When the changes are done click “SAVE” in the upper section on the right-hand side.