Statement on Web Crawling and Deep Indexing
Elsevier supports and encourages our institutional and corporate customers who want programmatic (i.e. automated) access to the content on ScienceDirect and other Elsevier Internet databases. We have developed complimentary services for our customers that are specifically designed to support machine-to-machine access, including a comprehensive suite of APIs and content syndication services. In order to ensure stability and maintain the performance of our user-facing platforms, we recommend that our customers use those Elsevier services to access content.
Elsevier has also put in place policies and technical measures to prevent crawling by robots or other mechanized devices. Web crawling is an inefficient method of harvesting large quantities of content from sites similar to our sites and imposes a high volume burden on our systems which interferes with access for human users in the research community. http://public.lanl.gov/herbertv/papers/f140-nelson.pdf).
To aid discovery for researchers, we also partner with major search engines such as Google and Google Scholar to enable them to index our content.
Customers who would liketo use Elsevier’s services to deep-index full text and bibliographic data on ScienceDirect, or other Elsevier Internet databases (for example to build indexes for the purpose of recreating navigational schemes to Elsevier content on a customer's or company's local servers), should contact their Account Manager for an overview of the commercial solutions we provide for this purpose.
We also recognise that some of our customers want to be able perform text or data mining within this content, and we encourage them to do so. To enable that functionality (and ensure that is not falsely identified as crawling), customersshould discuss content mining needs with their Account Manager so that they can utilize these services. Elsevier also has sophisticated services for delivering Journal metadata including bibliographic information and abstracts for the inclusion in 3rd party A&I databases, agents’ consolidated services, and others.
If you are a researcher, developer, or other end-user and you would like to use Elsevier data to build applications, perform text or data mining, or for other uses, please contact your institution’s library staff for assistance.
Top 25 Articles
Keep track of the latest trends in your subject area with ScienceDirect's Top 25 Hottest Articles. Visit top25.sciencedirect.com
Download the ScienceDirect mobile app today! Available for iPhone, iPad, Blackberry and Android devices.
News and insights for the research community. Visit elsevierconnect.com