How can I exclude certain pages from the search results?
There are 2 ways to exclude certain pages: Either with the setting "Regexes for forbidden URIs" or with a meta tag "noindex".
Regexes are used if an entire part (tree) of a webpage shall be excluded, or if links matching a certain pattern shall be excluded.
Examples:
To exclude everything in the path of www.mywebsite.org/contact/..., simply add @./contact/.@ to the forbidden URIs in the plugin settings
To exclude all links with an anchor you can add a very general regex like @.#.@
Meta tags can be used to tell the crawler not to index and/or not to follow links on this page:
Examples:
<meta name="pimcore_searchphp" content="noindex,follow"/> <meta name="pimcore_searchphp" content="noindex,nofollow"/> <meta name="pimcore_searchphp" content="index,nofollow"/>
Or you can use the more general tag targeting all robots:
<meta name="robots" content="noindex,follow"/> <meta name="robots" content="noindex,nofollow"/> <meta name="robots" content="index,nofollow"/>
How can I make certain pages more relevant/give them a higher ranking in the search results?
When the search term can be found in the <h1>Headline</h1> of a page - the page will have a higher ranking. This is currently the only reason why the boost factor of search result ist increased.
How can I get rid of those many overview pages in the search results? I want my detail pages to appear in search results only.
The crawler needs to go through all the overview pages of a list to find all detail links. Here is a common pitfall: You can not exclude those paging links by configuring "Regexes for Forbidden URIs"! If you did that the crawler would never go beyond the first page of your paged list. You only can get rid of overview pages in the search results if you have control over the content (which is the case in most use cases). Please equip your overview pages with a meta tag:
<meta name="pimcore_searchphp" content="noindex,follow"/>
Or to make it more general for all bots:
<meta name="robots" content="noindex,follow"/>
Doing that you tell the searchphp crawler not to index this page, but to follow all it's links.
Some of my detail pages are missing (e.g. news lists etc.)
Please make sure that the crawler can find all your detail pages. The following commonly known issues are reasons why the crawler can not find all detail links:
- Please check if the link depth in your SeachPhp settings is deep enough for the crawler to reach all paging links
- Does your list use random order? Please make sure that the items are not randomized again within the same browser session. The crawler processes share session cookies so they stick to one session, but the items may not be randomized differently when the crawler reaches the second page of your list.