Using Selenium with Python

Now we will write actual Python code to interact with a web browser.

Install Selenium Python module

The selenium module provides several useful methods that lets you browser web pages and interact with their web elements.

Some examples:

• find_element_by_id
• find_element_by_class_name
• find_element_by_tag_name
• find_element_by_css_selector
• find_element_by_partial_link_text
• find_element_by_xpath

For the following HTML fragment:

<div id="coolestWidgetEvah">...</div>
<div class="cheese"><span>Cheddar</span></div><div class="cheese"><span>Gouda</span></div>
<iframe src="..."></iframe>
<div id="food"><span class="dairy">milk</span><span class="dairy aged">cheese</span></div>
<a href="http://www.google.com/search?q=cheese">search for cheese</a>
<input type="text" name="example" />

We can use Selenium to locate several different elements:

element = driver.find_element_by_id("coolestWidgetEvah")
cheeses = driver.find_elements_by_class_name("cheese")
frame = driver.find_element_by_tag_name("iframe")
cheese = driver.find_element_by_css_selector("#food span.dairy.aged")
cheese = driver.find_element_by_partial_link_text("cheese")
inputs = driver.find_elements_by_xpath("//input")

For further information about the many more useful methods provided by Selenium, please refer to the documentation.

So let’s go ahead and install the selenium module:

Note

This step can be skipped if you’ve cloned the repository and installed all Python dependencies.

pip install selenium

Install a Web Driver

Selenium requires a driver to interface with the chosen browser. Firefox, for example, requires geckodriver Some of the more popular ones are:

Web browser	Download URL
Google Chrome	https://sites.google.com/a/chromium.org/chromedriver/downloads
Microsoft Edge	https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
Firefox	https://github.com/mozilla/geckodriver/releases
Safari	https://webkit.org/blog/6900/webdriver-support-in-safari-10/

Note

For this tutorial I have chosen to use the web driver for the Chrome web browser. On a Mac I can easily install it with:

brew cask install chromedriver

Note

Make sure to include the ChromeDriver location in your PATH environment variable

Interact with Chrome via Python Selenium

Open a Python shell and type the code below to search for Red Hat via Google interactively:

from selenium import webdriver
from selenium.webdriver.common.keys import Keys


# Start Google Chrome
browser = webdriver.Chrome()

# Open Google.com
browser.get('https://www.google.com/')

# Locate the search box
element = browser.find_element_by_id('lst-ib')
assert element is not None

# Search for Red Hat
element.send_keys('Red Hat' + Keys.RETURN)
assert browser.title.startswith('Red Hat')

# Visit Google.com
browser.get('https://www.google.com')

# Locate the Google Logo
element = browser.find_element_by_id('hplogo')
assert element is not None

# Change the logo and adjust its height
browser.execute_script(
    "arguments[0].setAttribute('srcset', "
    "'https://omaciel.fedorapeople.org/ogmaciel.png')",
    element
    )
browser.execute_script(
    "arguments[0].setAttribute('height', '100%')",
    element
    )

# Close the browser
browser.quit()