How To Extract Booking.com Data For Hotels?

• Hotel’s Name
• Location
• Room Type
• Pricing
• Pricing For (eg: 2 Adults, 1 Night)
• Overall Ratings
• Bed Type
• Total Reviews
• Rating Tile
• Links

Installing the Packages Required to Run a Booking Data Scraper

We would require these Packages of Python 3

• Python Requests to do requests as well as downloading HTML content through Search Result pages from Booking.com.
• SelectorLib Python suites to extract data with YAML files that we have made from webpages, which we download.

Make installation using pip3

pip3 install requests selectorlib

The Code

It’s time to make a project folder named booking-hotel-scraper. In this folder, add one Python file named scrape.py

After that, paste the code given here in scrape.py

from selectorlib import Extractorimport requests from time import sleepimport csv# Create an Extractor by reading from the YAML filee = Extractor.from_yaml_file('booking.yml')def scrape(url):    headers = {'Connection': 'keep-alive','Pragma': 'no-cache','Cache-Control': 'no-cache','DNT': '1','Upgrade-Insecure-Requests': '1',# You may want to change the user agent if you get blocked'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.113 Safari/537.36','Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Referer': 'https://www.booking.com/index.en-gb.html','Accept-Language': 'en-GB,en-US;q=0.9,en;q=0.8',}# Download the page using requestsprint("Downloading %s"%url)r = requests.get(url, headers=headers)# Pass the HTML of the page and create return e.extract(r.text,base_url=url)with open("urls.txt",'r') as urllist, open('data.csv','w') as outfile:fieldnames = ["name","location","price","price_for","room_type","beds","rating","rating_title","number_of_ratings","url"]writer = csv.DictWriter(outfile, fieldnames=fieldnames,quoting=csv.QUOTE_ALL)writer.writeheader()for url in urllist.readlines():data = scrape(url) if data:for h in data['hotels']:writer.writerow(h)# sleep(5)

This code will:

Open the file named urls.txt as well as download HTML content given for every link in that.

Parse this HTML with Selectorlib Template named booking.yml

Then save the output file in the CSV file named data.csv

It’s time to make a file called urls.txt as well as paste the search result URLs in it. Then we need to create a Selectorlib Template.

Make Selectorlib Template for Scraping Hotels Data from Booking.com Searching Results

You may notice that within a code given above, which we used the file named booking.yml. The file makes this code so short and easy. The magic after making this file is the Web Scraping tool called Selectorlib.

Selectorlib makes selecting, marking, as well as extracting data from the webpages visually easy. A Selectorlib Web Scraping Chrome Extension allows you to mark data, which you want to scrape, and makes CSS Selectors required for extracting the data. After that, preview how the data could look like.

In case, you require data that we have given above, you should not use Selectorlib. As we have already done it for you as well as produced an easy “template”, which you may use. Although, if you need to add new fields, you may use Selectorlib for adding those fields into a template.

Let’s see how we have moticed the data fields we needed to extract with Chrome Extension of Selectorlib.

When you have made the template, just click on ‘Highlight’ button to highlight and preview all selectors. In the end, just click on ‘Export’ option and download YAML file, which is a booking.yml file.

Let’s see how the template – booking.yml will look like:

hotels:css: div.sr_itemmultiple: truetype: Textchildren:name:css: span.sr-hotel__nametype: Textlocation:css: a.bui-linktype: Textprice:css: div.bui-price-display__valuetype: Textprice_for:css: div.bui-price-display__labeltype: Textroom_type:css: strongtype: Textbeds:css: div.c-beds-configurationtype: Textrating:css: div.bui-review-score__badgetype: Textrating_title:css: div.bui-review-score__titletype: Textnumber_of_ratings:css: div.bui-review-score__texttype: Texturl:css: a.hotel_name_linktype: Link

Run a Web Scraper

For running the web scraper, from a project folder,

1. Try to search Booking.com to see your Hotels requirements
2. Copy as well as add search results URLs into urls.txt
3. Then Run the script python3 scrape.py
4. Find data from the data.csv file

Let’s take a sample data from the search results pages.

Contact Web Screen Scraping if you want to Extract Booking.com Data for Hotels!