• Current version

Webmasters


Last modified: September 12, 2012

Getting your .XXX sites crawled

Our search engine will ONLY crawl developed .XXX websites that contain adult entertainment content. It is important to ensure that you have your membership process completed and can successfully navigate to your website via your browser. If you can find the .XXX website via your browser then our search engine will be able to find it too.

We limit the crawler to .XXX sites only. If your content links to other sites in other TLDs then our crawler stops following these links once it comes across a link that does not end in .XXX.

Developed Sites

Only sites that have developed content are included in the crawler. If your site is being hosted at a parked service provider then it may not be crawled. Please contact support if you feel your site has been overlooked or not crawled.

Robots.txt File

The crawler will respect the robots.txt file. You can use this file to restrict access to your site by the search engine crawler. We recommend that you use a robots.txt file to exclude any pages that users would not generally be able to access; e.g. pages that require login or membership.

Virtual Servers

If you have an adult entertainment .XXX domain name that you are forwarding to an adult entertainment site located in a .com (.net .org etc…), check that your DNS uses an “A” record to point your .XXX to your web servers IP address (Do Not Use a CNAME). In your web server configuration put the .XXX domain that you want the web server to answer for - see example below:

<VirtualHost 192.168.0.1>
      ServerAdmin webmaster@domain.com
      DocumentRoot /usr/local/apache/htdocs/site_1
      CustomLog logs/Site.log common
      ServerName *.domain.com
      ServerName *.domain.xxx
</VirtualHost>


In the case of multiple domains (including other TLDs) to the same site – please ensure that relative links are used within the site to allow our crawler to navigate through them.

Forwarding Your URL with Masking

If you use your .XXX domain to forward to another domain, we will only continue to follow the search results if the URL is being masked as well. If the URL is not being masked, your site will only continue to be crawled if it ends with .XXX. Contact your DNS provider or Internet Registrar for more information on how to forward and mask your .XXX URL.

Relevance of Your Rankings

Many factors go into our search results rankings. Some factors to consider when looking at your results within our search engine:

  • The appearance of the search term in the domain name, header, title, URL, content on the page ultimately ensures the web page will be considered for display.
  • The number of words on the webpage itselt that describe the content (video, image); if there are a small number words then the web page may not get crawled.
  • The relevance of search results will change over time with popular (more) frequent links increasing your rankings.

Search Opt-Out

If you prefer not to be listed on www.search.xxx, please let us know by sending an email to support. We will verify your request with the information we have in your Membership Application to ensure it is the proper party requesting this opt-out. Once verified, we will complete your request.