# Overview of Parallel Web Systems’ Bots

Parallel uses robots and user agents to gather public web content for its products and to retrieve web content at users' direction. Parallel publishes named user agents so webmasters and content owners can manage and identify how their content is accessed. Below is information on how Parallel accesses web content:

## **ShapBot**

ShapBot collects publicly available web content to build and maintain Parallel's search index, maximizing site visibility in search results. Restricting ShapBot prevents our system from indexing your content, which may reduce your site's visibility and discoverability through Parallel's web APIs and search features. Webmasters can manage automated access using robots.txt with the User-agent: ShapBot directive.

_Full user-agent string: Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0_

For the complete list of ShapBot IPs, see shapbot.json[shapbot.json](https://docs.parallel.ai/resources/shapbot.json).

## **Shap-User**

Shap-User identifies itself when accessing content on behalf of users, giving content owners visibility into user-initiated requests. Shap-User is not used for crawling the web in an automatic fashion. Shap-User is intended to provide visibility rather than to be managed through automated controls. To manage automated crawling, see ShapBot above.

_Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; Shap-User/0.1.0_

## **Contact & Support**

If you believe that ShapBot may be malfunctioning, or need assistance, please contact us at support@parallel.ai[support@parallel.ai](mailto:support@parallel.ai). Please reach out from an email that includes the domain you are contacting us about, as it is otherwise difficult to verify reports.

## **Content Inquiries & Partnerships**

Content owners with questions about our search index, or who would like to explore joining Index by Parallel, can reach us at index@parallel.ai[index@parallel.ai](mailto:index@parallel.ai).