I have salesforce community (salesforce site) and have to index that site by google.
How can I index my site by google? Ex:- https://Mywebsite.com/
I have to allow few pages only(not all pages), how can I do this ?
I have some user profiles which is a public page. it can be with query string parameter. How to index those user profiles by google ? Ex:- https://Mywebsite.com/apex/UserProfile?id=001K000000sLyjhIAC
This id will be changed based on the user profile .
Go to Setup > Develop > Sites. There should be an entry there corresponding to your community (there's a 1-1 relationship between a Site and a Community)
First create a Visualforce page containing (for example) the following:
<apex:page contentType="text/plain"> User-agent: * Allow: / </apex:page>
Choose this page on the
Site Robots.txt attribute of the Site associated with your Community.
Then confirm you can see its contents when you visit your community at https://mywebsite.com/robots.txt
User-agent: * Allow: /
A bit more reading on how robots.txt works will help you allow/disallow the appropriate pages. Anything behind a login cannot be crawled, and those custom user profile pages must be discoverable via links.Tweet